ChatGPTにChatGPTの限界を聞いてみよう

まるで知識豊富な人間と相対しているかのような会話が楽しめる人工知能ChatGPTが人気だ。実用性もあるように思え、東大の松尾研の人が「検索がなくなる」「Office製品は全部変わる」「ホワイトカラーの仕事のほとんどすべてに何らかの影響がある」かも知れないと評価した話が広まっている。松尾研は誇大広告と言うか未来予想的な話をするので^*1、もうちょっと地に足がついた評価が欲しい。そこで、ChatGPTにChatGPTの限界を聞いてみた。

まず、人間相手でもそうだが、漠然とした質問よりはより具体的な比較がある方が回答の情報が豊富になりやすいので、大昔からあるエキスパート・システムとの違いを聞いてみた。

What is the most significant difference between chatGPT and expert systems? — ChatGPT and expert systems are both examples of artificial intelligence technologies, but they differ in significant ways.

ChatGPT is a large language model that is trained on vast amounts of text data, allowing it to generate human-like responses to a wide range of questions and prompts. It uses natural language processing (NLP) techniques to understand the context and meaning of text and to generate responses that are coherent and relevant to the user's input. Unlike expert systems, which are typically designed for a specific domain, ChatGPT can respond to questions and topics across a wide range of domains and subject areas.

Expert systems, on the other hand, are computer programs designed to mimic the decision-making abilities of a human expert in a specific domain. They typically use a set of rules or algorithms to analyze data and make decisions based on that analysis. Expert systems are often used in fields such as medicine, finance, and engineering, where their ability to process large amounts of complex data can be useful in making decisions.

The most significant difference between ChatGPT and expert systems is that ChatGPT is a general-purpose language model that can respond to a wide range of questions and topics, while expert systems are designed for a specific domain and are typically focused on decision-making. ChatGPT is designed to simulate human conversation and generate human-like responses, while expert systems are focused on analyzing data and making decisions based on that analysis.

エキスパート・システムは意思決定のためのデータ分析を目的としている一方で、ChatGPTは人間の会話と人間的な応答を模倣するように作られているそうだ。

学習データを厳選したらエキスパート・システムと同様に使えるのではないかと言う疑問が出てきたので、聞いてみた。

Can ChatGPT be a domain-specific if it learns specific datasets of text? — Yes, ChatGPT can be fine-tuned on domain-specific datasets of text, which would allow it to generate more accurate and relevant responses in that specific domain. This process of fine-tuning involves training the model on a specific dataset, which could be related to a particular domain such as finance, healthcare, or legal.

By fine-tuning the model on a domain-specific dataset, ChatGPT can learn the specific language and terminology used in that domain, as well as the context and patterns of conversation within that domain. This would enable ChatGPT to generate more accurate and relevant responses when it comes to the specific domain in question.

However, it's important to note that even when fine-tuned on a specific domain, ChatGPT is still a general-purpose language model and may not be as specialized as a dedicated expert system designed specifically for that domain. Expert systems typically incorporate domain-specific knowledge and rules to make more precise decisions in a given domain, while ChatGPT relies on its ability to generate responses based on patterns and relationships learned from vast amounts of text data.

使えると言いつつ、精確な意思決定を目的として知識とルールを与えられているエキスパート・システムとつくりが違うと念をさされる。

今は劣っても、将来的には精確な意思決定のための知識とルールを学習できるようになるのではないかと聞いてみた。

Will ChatGPT be improved to get domain-specific knowledge and rules to make precise decisions from specific datasets of text in the future? — There is ongoing research and development in the field of natural language processing (NLP) that aims to improve the ability of language models like ChatGPT to incorporate domain-specific knowledge and rules. One approach is to combine language models like ChatGPT with other techniques, such as knowledge graphs, to enable the model to understand and reason about domain-specific concepts and relationships.

Additionally, there are techniques such as "prompt engineering" that involve designing specific prompts or inputs to the model that encourage it to generate responses that are more aligned with a particular domain or task. This can be used to fine-tune the model to a specific domain or to prompt it to generate responses that are more focused on a particular task.

As research in NLP continues to advance, it's likely that we will see further improvements in the ability of language models like ChatGPT to incorporate domain-specific knowledge and rules, as well as the ability to make more precise decisions based on specific datasets of text. However, it's important to note that language models like ChatGPT are still primarily focused on generating responses based on patterns and relationships learned from vast amounts of text data, rather than on incorporating explicit rules and knowledge specific to a particular domain.

言語モデルと知識グラフを組み合わせるような自然言語処理の研究開発もされているし、もっと即席的にチューニングする技術も存在するが、ChatGPTはまだテキストデータのパターンと関連に基づいた応答に注力しており、明示的に知識とルールを取り込むことは目指していないそうだ。

さて、さらっと大嘘を混ぜてくると警告されているChatGPTによるこのChatGPTの説明、信じてよいものであろうか？

*1関連記事：人工知能×布教文学×ポストモダン