![]() Inspired by the Meta LLaMA and Stanford Alpaca project, we introduce Vicuna-13B, an open-source chatbot backed by an enhanced dataset and an easy-to-use, scalable infrastructure. However, despite its impressive performance, the training and architecture details of ChatGPT remain unclear, hindering research and open-source innovation in this field. The rapid advancement of large language models (LLMs) has revolutionized chatbot systems, resulting in unprecedented levels of intelligence as seen in OpenAI's ChatGPT. ![]() Relative Response Quality Assessed by GPT-4* Online Demo More details are provided in the evaluation section.įigure 1. While this proposed framework shows a potential to automate chatbot assessment, it is not yet a rigorous approach.īuilding an evaluation system for chatbots remains an open question requiring further research. Preliminary evaluations based on GPT-4, summarized in Figure 1, show that Vicuna achieves 90% * capability of Bard/ChatGPT. Our initial finding indicates that GPT-4 can produce highly consistent ranks and detailed assessment when comparing chatbots’ answers (see above example of GPT-4 judgment). With recent advancements in GPT-4, we are curious whether its capabilities have reached a human-like level that could enable an automated evaluation framework for benchmark generation and performance assessments. However, evaluating chatbots is never a simple task. How Good is Vicuna?Īfter fine-tuning Vicuna with 70K user-shared ChatGPT conversations, we discover that Vicuna becomes capable of generating more detailed and well-structured answers compared to Alpaca (see examples below), with the quality on par with ChatGPT. *According to a fun and non-scientific evaluation with GPT-4. ![]() Vicuna (generated by stable diffusion 2.1) The code and weights, along with an online demo, are publicly available for non-commercial use. ![]() The cost of training Vicuna-13B is around $300. Preliminary evaluation using GPT-4 as a judge shows Vicuna-13B achieves more than 90%* quality of OpenAI ChatGPT and Google Bard while outperforming other models like LLaMA and Stanford Alpaca in more than 90% * of cases. We introduce Vicuna-13B, an open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |