Artificial intelligence technology, which had been undergoing a "glacial period" due to questions about slow technology development and effectiveness, will reach its second heyday thanks to the development of deep learning technology in the mid-to-late 2010s.
The image processing sector, which has drawn attention due to the development of convolutional neural networks (CNN), including Resnet, is used in various fields, regardless of industry and industry, and people hope to predict tomorrow's stock price using deep learning as quantities in the financial industry use RNNs (Recurrent Neural Networks) for traditional time series analysis.
However, even in the heyday of deep learning, it wasn’t quite as popular as the NLP (natural language processing) field. Human language has ambiguities and irregularities that can’t decipher 100% of the speaker's intention, as well as the characteristics that the meaning changes depending on the position and nuance of the word in the sentence. Because of these characteristics, the field of natural language processing has been shunned by many.
1. The Limitations of NLP
As the technical limitations of NLP, a long-term dependence problem for traditional RNN and LSTM models, were overcome by a paper published by Google in the 2017s called 'attention is all you need', NLP research faced an inflection point. Since then, super-large language models (LLMs) have begun to be developed one after another amid the trend of enlargement of language models and now the heyday of natural language processing (NLP) has finally begun. In this session, let's learn about language models and natural language processing that have already penetrated deeply into our daily lives.
2. What is Natural Language Processing?
Can't we teach computers the 'language' that has long been considered exclusively to humans, and furthermore, let them use the language freely on their own? Long-standing studies to answer these questions have come together to create a field called Natural Language Processing (NLP).
In short, NLP (natural language processing) refers to a field of machine learning (deep learning), that deals with all techniques that allow computers to process and apply all non-artificial, naturally occurring languages (Korean, English, Japanese, etc.).
Key Components of NLP
NLP can be divided into two main areas: Natural Language Understanding (NLU) and Natural Language Generation (NLG).
Natural Language Understanding (NLU)
NLU refers to the processes that allow computers to understand the grammar and context of each language text as it is intended to be.
Natural Language Generation (NLG)
On the other hand, NLG deals with creating content that humans can directly understand, based on what the computer comprehends.
Applications of NLP
NLP finds utility in numerous technologies, including:
Translators: Tools that convert text or speech from one language to another.
Chatbots: Programs designed to simulate conversation with human users, especially over the internet.
Text Analysis: Techniques to extract meaningful information from text data.
These technologies typically mix elements of both NLU (input) and NLG (output).
3. NLP On Display in Society
Dataization of text has revolutionized various fields, with NLP finding numerous applications in daily life. Most people have used or benefitted from NLP in areas such as chatbots, rating analysis, translation, and text analysis.
Real-Life Examples of NLP
One significant example of NLP in action can be observed in the review and rating analysis of food delivery services. User-generated content like these reviews transforms into valuable text data that platforms can analyze.
Review Analysis
Review analysis not only offers insights into customer sentiments but also uncovers detailed opinions. For instance, it can reveal whether customers find the food salty, the portion sizes small, or the delivery time slow.
Improving Business Practices
Based on these insights, business owners can make necessary improvements to their services. Moreover, delivery platforms can use this information to recommend stores that align with customer preferences, thereby enhancing the overall user experience.
Personalized Recommendations and Offers
By harnessing the power of NLP, platforms can provide personalized recommendations, issue tailored coupons, and create a more engaging customer experience.
4. How a Computer Understands NLP
Computers, in their fundamental operations, recognize data in the form of numbers - primarily zeros and ones. Regardless of whether the data is in the form of voice, image, or text, a computer processes it numerically.
NLP and Numerical Data
Consider an iPhone's Siri understanding a question in 'natural language' and responding with the weather forecast instead of merely repeating the query "Today's weather?" This is only possible because the natural language query has been transformed into numerical data that the computer can comprehend, a process known as vectorization.
NLP Preprocessing Steps
To facilitate this transformation and ensure that the machine learning model functions optimally, several preprocessing steps are required:
Tokenization: This breaks down sentences into meaningful units or words.
Padding and Normalization: These steps help prevent information loss depending on the model.
Stop Word Designation and Categorization: These preprocessing processes are necessary for the model to perform better.
The Role of Machine Learning in NLP
Once the data is prepared, a machine learning (specifically, deep learning) model can be developed. There are two main types of models that enable NLP:
Statistical Language Models
Neural Network-Based Transfer Learning Models
This category includes models such as Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN). Transfer learning models, which have undergone extensive pre-training, typically outperform statistical language models and are widely used in the current NLP landscape.
The Rise of NLP
The resurgence of artificial intelligence is largely driven by the advances in deep learning and natural language processing (NLP). Technologies like convolutional neural networks have revolutionized image processing, while recurrent neural networks have found wide applications in industries such as finance.
Yet, it is in the field of NLP that deep learning truly shines. Overcoming previous technical limitations, the NLP field has seen a meteoric rise with the development of large language models that can understand and generate human language, bringing a second heyday of AI.
Despite the complexity and ambiguity inherent in human language, breakthroughs in NLP technology have made it possible for machines to not only understand our words but also context and intent. Through the amalgamation of Natural Language Understanding (NLU) and Natural Language Generation (NLG), NLP has given birth to various applications such as chatbots, translators, and text analysis systems that have deeply permeated our everyday life.
