The requirement is to build a web application that evaluates the responses of different LLM models in the Question and Answering task. The models to compare are as follows

gpt$-3\mathrm{.}5-$turbo

gpt$-4$

Llama$-2-70$b$-$chat

Falcon$-40$b$-$instruct

Integrate a vector database into the application. The vector database needs to store information scraped from the websites provided.

When the user enters a prompt, query the vector database to retrieve search results based on the user's prompt, get all relevant results and then query each of the $4$ LLMs with the new prompt that contains the search results and the user's initial query.

The LLMs should be able to reference the search results and respond with an answer to the user's initial query. The prompt should be engineered in such a way that the model should only reference the search results to construct the response and not its own knowledge.

Please note the backend application needs to be in Python. Usage of streamlit is not allowed.

The primary objectives are as follows:

Design and develop a web application that accepts user prompts and provides the response of the LLMs$.$

Implement a scraping logic to scrape the mentioned website, and store the information in a vector database.

When a new user prompt is received, Query all the mentioned LLMs with the search results from the vector database and the user's query. And then send this output to the frontend.

Implement a mechanism to compare and evaluate the generated outputs to find the best$-$performing LLM for the given user input.

Implement Websockets or Server$-$Sent Events $($SSE$)$ to enable real$-$time streaming of responses to the frontend.