While I was wondering what I should write about in my last blog, I realized how much I actually enjoyed writing. As I am considering doing a minor in journalism – and even masters -, I searched online for future expectations if I were to become a journalist. Regrets. I should not have done that. I stumbled on an article by the NRC, about a new AI development called GTP-3. Thanks to this text generator, writers might not be necessary in the future anymore.
What is GTP-3?
Wait what? Yes: you’ve read this correctly. The AI-writer GTP-3, Generative Pre-trained Transformer 3, developed by OpenIA, is the outcome of the refinement of its two previous versions that are build last year in 2019. GTP-3 is the largest AI neuronetwork until now, containing 96 layers of neurons and 175 billion parameter connections. This network is trained by 45 terabyte of data, which consist of the English Wikipedia, all texts that are available online and written between 2016 and 2019 and countless books. Due to the artificial intelligence neuronetwork and the broad range of texts, the AI-writer is able to recognize certain text constructions.
What is so special?
So what is this robot capable of? Even though it needs a lot of fine-tuning, the robot can already perform human writing tasks. As it does not have a human brain – well ofcourse – it is not capable of comprehensing what words mean. However, due to its training, GTP-3 can calculate what words are propable to match in sentenses, what a follow-up sentence could be, and these sentenses will form a text. In addition, it is also able to complete stories and correct grammar mistakes. It can answer language related questions with only a few examples translate words. Moreover, mathematical problems can be solved – till some extend though. Until now, there has not been such a multifunctional language generator as GTP-3.
Improvements
As I mentioned already, there is still room for improvements. Even though it is provided with an incredible amount of data, it is limited in processing humanlike thoughts. More data equals a better understanding of language, but not it does not really become smarter. According to Gary Marcus, the founder of Robust.AI, GTP-3 has just as its predecessor problems with “biological, physical, psychological and social reasoning“. Therefore, the system is not reliable in many cases. Moreover, OpenAI included a section concerning gender, race and religion in their scientific article. Namely, GTP-3 is trained through texts that may be outdated in terms of stereotyping behavior. It is therefore not surprising that the AI model will reflect these stereotypes that are present in their training data.
In sum, GTP-3 is a very interesting development, but it is definitely well-trained enough.
To go back to my future career, GTP-3 will not replace the job of a journalist. Yet, it might be very possible that a future language generator – perhaps GTP-3’s ‘grandchild’ – might get closer to a human-like brain. Until then, we are still better than robots in writing and I will pursue my aspirations to follow some journalism courses. Thank you Digital Media, Culture, and Society for not only teaching me about the Tech, but about my (uncertain) future career as well.
Sources:
- https://arxiv.org/pdf/2005.14165.pdf
- https://www.nrc.nl/nieuws/2020/10/23/de-nieuwste-ai-robot-is-onbetrouwbaar-gammel-en-bewonderenswaardig-a4017171
What do you think of these developments?
Thanks for this interesting blog! I also think that we’re still far away from robotic authors because they still lack in emotions and writing style. Furthermore I really like authors that use (very) specific words that we don’t use every day in normal communication. I think that the machine is not able to do that too now, but maybe that will change in 20 or 30 years. More important is the fight against fake news and therefore I think it is important to keep human authors in business so that they can prove their truth. For us as reader it is, in that case, still possible to check the author before relying on him/her/it(?). For that reason, It is guaranteed that you will find a job after your studies!
I’m glad that you found that your future career won’t be taken over by AI! I’m worried that this GTP-3 or similar algorithms will be misused as it could spread huge amounts of misinformation. Just like automated social media bots were used to polarize political opinions in the US before the election, I could imagine that AI articles would also have some sort of alternate agenda. On the other hand, I read an article saying that the algorithm itself was the best at detecting fake news/AI generated articles – so it could, technically, also be used for something good. I still definitely believe that humans are needed in journalism, especially for the creative and compassionate components of the job, even though the writing itself is centered on non-fiction.
I agree with Therese, I’m also glad to hear that your future job will be still done by humans 🙂
It is very interesting to observe the day-to-day developments of language related AI/digital technologies. Since Japanese has super different grammar than Indo-European language family, it is a very complicated task for AI to translate Japanese into English or translate English into Japanese. So, Japanse generally have bad impressions on translation softwares, and so did I. However, I recently discover the software called DeepL Translation and it writes more or less correct Japanese, and I was really surprised. Also, I remember that there is a project to make AI write a novel (with their learned grammar styles of writers) and the work of the AI actually went through the first step of a famous short-short novel contest. As these shows, the advancement of language related digital technology today is really impressive and I think the general seamless communication skill itself would be attained by AI soon enough. However, I really think that poetry or beautiful metaphors, which are the privilege of humanities, cannot be impressively newly invented by AI however they learn the patters of successful poetries in the history because they are the products of humans’ sensory observations of the world. I hope the advancement of the language related technologies from now on will be accompanied by people’s shift of focus to the poetry and the dreamy use of languages.
Thanks for your nice article!!
I find it’s very interesting how a lot of people in our generation face this fear of being replaced by the AI in our future jobs, even though these technologies are being developed to make our future easier! I recently had conversation with a few of my friends, who all chose different paths, but all were concerned about this issue, to some extent. However, I agree with you, there is still a long way to go for technologies to replace humans. Moreover, I believe some jobs will always require human participation