Understanding Deployment Tools for Large Language Models in Real-World Applications: An Empirical Study

Bachelor & Master Thesis

Large language models have revolutionized various fields, yet their deployment into real- world applications presents challenges. This bachelor's thesis aims to conduct an empirical study focusing on the issues, challenges, and future prospects of deploying large language models using open-source tools. The project involves a systematic analysis of these tools, their functionalities, and their effectiveness in facilitating the integration of language models into diverse applications.

What you need to do in this project is to:

• Identify and compile a comprehensive list of open-source deployment tools designed for large language models.
• Create a systematic methodology to crawl and analyze issue reports from the repositories of these identified tools.
• Classify and categorize the encountered issues and feature requests based on severity, frequency, and nature.
• Perform a comparative analysis of the tools based on their performance, ease of use, community support, and adaptability to various application types.
• Propose recommendations and potential improvements for the identified tools to address prevalent issues and enhance their usability.

Skills Required:

• Good programming skills, like Python
• Great summarization, presentation and writing capability

Note that this project can be carried out remotely. Students with great performance may be granted an opportunity to do a paid Hiwi in the coming semester break and even a PhD position in the future.

Reference:

github.com/Hannibal046/Awesome-LLM;

https://github.com/lm-sys/FastChat/issues?q=is%3Aissue+is%3Aclosed