Abstract:

This video demonstrates how to implement real-time streaming of OpenAI responses within a Django web application. It covers both the backend and frontend components necessary to achieve a word-by-word display of AI-generated content, rather than waiting for the complete response. The tutorial details the creation of a Python generator function to interface with the OpenAI API's streaming capability and leverages Django's StreamingHttpResponse to maintain an open connection. On the client side, JavaScript's fetch API, ReadableStream reader, and TextDecoder are used to process and incrementally display the incoming text. Furthermore, the video shows how to integrate marked.js to parse and render Markdown formatting in the streamed responses, enhancing the user experience. The presenter highlights that while this method is suitable for small applications, larger deployments might benefit from ASGI for non-blocking I/O.

Streaming OpenAI Responses in a Django App

• 0:00 Introduction: The video introduces the goal: streaming OpenAI responses in a Django app for a word-by-word display, demonstrating with examples like "How are you?" and "Describe Python in 500 Words."
• 0:43 Initial Setup: The presenter outlines the existing project structure, including views.py with an imported OpenAI client and API key, urls.py with an index path, and index.html containing a basic form for user input.
• 1:33 Creating a Python Generator (generate_response):
  • A function generate_response is defined to take a question.
  • It uses client.chat.completions.create with model="gpt-4", a user message containing the question, and crucially, stream=True.
  • The function iterates through chunks from the stream and yields chunk.choices[0].delta.content if new content is available, making it a Python generator.
• 5:14 Django answer View (StreamingHttpResponse):
  • A new Django view answer is created, importing StreamingHttpResponse from django.http.response.
  • It's decorated with @csrf_exempt to allow POST requests without CSRF tokens.
  • Initially, a hardcoded message is used to test the streaming.
  • The view returns StreamingHttpResponse by passing the generate_response generator, a status=200, and content_type="text/plain".
  • A demo in the browser shows the streaming, although the initial words might be missing due to browser behavior.
• 8:00 Frontend Integration (JavaScript):
  • In index.html, an async click event listener is added to the submit button.
  • e.preventDefault() is used to prevent default form submission.
  • A fetch POST request is sent to the /answer/ endpoint with the user's input as a JSON message.
  • 10:38 Backend Update: The answer view is updated to parse the incoming JSON request body using json.loads(request.body) to extract the user's message.
• 11:11 Handling Stream on Frontend:
  • The JavaScript code retrieves a reader from the response.body.
  • It enters a while (true) loop to continuously read chunks.
  • const {done, value} = await reader.read() fetches the next part of the stream.
  • output += new TextDecoder().decode(value) decodes the chunk and appends it to an output string.
  • body.innerHTML = output updates the display with the accumulated text.
  • The loop exits if (done) is true, indicating the stream has ended.
• 13:27 Full Demo: A live demonstration confirms that user input now generates streamed AI responses displayed progressively on the same page.
• 14:03 Markdown Support:
  • The marked.js library is introduced by adding its script tag.
  • The body.innerHTML update is changed to body.innerHTML = marked.parse(output) to interpret and render Markdown syntax in the streamed response.
  • A demo shows an AI response formatted with bold text and lists, successfully rendered by marked.js.
• 15:25 Conclusion: The video concludes by reiterating the simplicity of the streaming method for small applications, advising ASGI for larger apps to avoid blocking users, and inviting questions and feedback.