The Next Technological Time Warp

Innovation The Next Technological Time Warp Dave Wright Brand Contributor ServiceNow BRANDVOICE Storytelling and expertise from marketers | Paid Program Aug 24, 2022, 08:26pm EDT | Share to Facebook Share to Twitter Share to Linkedin Developments in AI and machine learning are accelerating every day, but just how fast? Moore’s Law is the principle that the speed and capability of computers can be expected to double every two years. In economic terms, this means that at a constant dollar, your compute capacity will double about every 18 to 24 months. AI has us doing the time warp again—and and again and again.

getty But in the age of AI, Moore’s Law itself might become obsolete. In fact, machine learning makes it possible for computational capacity to grow up to 16x over the same two years. Yes, AI is moving that fast.

In light of these developments, I sat down with Nicolas Chapados, VP of research for ServiceNow’s advanced technology group, to discuss current AI-driven language modeling and image generation trends—before they become old news. Dave Wright: You and your team look at trends and new research papers. A while back, we were hearing a lot about GPT-3 and its concept as a large language model.

Do you mind filling in our readers on what exactly a large language model is? Nicolas Chapados: Simply put, a language model like GPT-3 is a statistical model of which words are likely to follow each other in a given language, like English. As an example, the word “service” is likely to be followed by words such as “agent” and “delivery,” but not a word like “cactus. ” So, these language models have been around for decades and are a key building block of natural language processing applications, speech recognition engine, machine translations, and so on.

Natural language modeling has attracted the most interest from the AI research community over the last few years. As a result, a big chunk of resources has been invested globally in advancing these capabilities, and for good reason—natural language processing is the building block for all modern speech recognition, document processing, translation, and document summarization. Researchers lately have been asking, “What if we make the models bigger and train them longer?” They discovered that growing the model gives it a better grasp of natural language, allowing it to have more context into patterns, which leads to more accurate results in, say, a speech recognition engine.

Wright: So, what was the big step forward with GPT-3? Was it the volume and size of the data used to train it? Or was this a step forward in algorithmic technology and what it was doing with the data? Chapados: What’s surprising is that the step from GPT-2 to GPT-3 included few if any changes in algorithm. It was purely a matter of scaling things up, making the models larger and training them with more data. For instance, GPT-2 had about 1.

5 billion parameters. GPT-3 has more than 100 times as many, with 175 billion parameters. As the parameters were scaled up, so was the amount of data used to train the models.

This required huge computational resources, including hundreds of GPUs working in parallel for months. Despite the effort to train these models, what is shocking is how much scale can enhance a model’s output and functionality—which is essentially the difference between GPT-2 and GPT-3. Wright: Regarding AI-generated fake news, could AI detect whether AI created something? Chapados: Fake news detection is a huge topic.

There is no definitive answer to this right now because fake news is simply another piece of text. Additionally, earlier language models, like GPT-2, could create cohesive sentences across a few paragraphs, but the cohesiveness would degrade in longer text examples. Now, with more complex models, text and images will remain cohesive for a long time.

Uncovering whether a human or a machine writes it is getting increasingly difficult . Even if you look at other modalities, like early image generation models and their ability to generate faces—not to mention today’s image generation capabilities with models such as DALL-E 2—telling the difference between human-generated and AI-generated content has been difficult for quite some time now. Wright: What do you make of this new technology? Chapados: DALL-E 2 came out recently and is notable for what it generates based on input text.

For example, researchers asked DALL-E to generate an image with the prompt “A woman at a coffee shop working on her laptop wearing headphones, painting by Alphonse Mucha”—which doesn’t exist anywhere. But the image created is indeed that of a woman, with a laptop and headphones, in a historical style reflective of Alphonse Mucha’s art. We are seeing incredible capabilities in creating a dialogue between a human and a machine to generate a creative output.

The images are in fact the result of a very fancy random number generator. Although access to DALL-E is restricted to avoid the proliferation of toxic imagery, we are seeing incredible capabilities in using it to create a dialogue between a human and a machine to generate a creative output. This could perhaps lead to the rise of virtual agencies.

Wright: Consumer AI is ubiquitous. Do you think that there’s a big gap between consumer AI and enterprise AI? Chapados: Yes, consumer use cases for AI are a few years ahead of enterprise. The teams spending the most on AI R&D are consumer brands: Meta, Google, etc.

However, consumer experiences, largely driven by AI, will shape the expectations of enterprise AI , especially in terms of customer experience—leaving the enterprise not too far behind in this regard. Moreover, some startups are building AI tools focused on generative work outputs for things like social posts, emails, documents, and even artwork to create new designs. So you could have, for example, a system that takes an idea for a design as the input and generates an asset in your marketing team’s look and feel.

There are more emerging AI examples for the enterprise, but they are still in their early stages of development. In my opinion, we are just starting to grasp the various applications of generative AI and what it can do in various enterprise use cases. Dave Wright Editorial Standards Print Reprints & Permissions.