This video provides a developer-focused walkthrough of the Sercale platform, a cloud service partner of Qualcomm that runs the Qualcomm AI Inference Suite on AI 100 Ultra accelerator cards. Presented by Ray Stevenson from Qualcomm's developer relations, the tour demonstrates the platform's user interface and core functionalities. Key features showcased include a multi-model chat interface with access to various LLMs like Llama 3.1 (8B and 70B), fast image generation using SDXL Turbo, and a text embedding service. The platform offers comprehensive developer resources, including a Python SDK and a REST API with embedded code samples and tutorials. Additionally, it provides tools for managing models, monitoring usage with rate limits and pricing information, and exploring agentic AI capabilities through examples like a stock analysis agent.
A Developer's Guide to the Sercale Platform Powered by Qualcomm AI
00:00:13 Introduction to Sercale: Ray Stevenson introduces the Sercale platform, a Qualcomm partner service that provides cloud access to the Qualcomm AI Inference Suite running on AI 100 Ultra inference cards.
00:01:06 Multi-Model Chat Interface: The platform features a chat interface where developers can select from various installed Large Language Models (LLMs), such as the fast Llama 3.1 8B model and the more powerful 70B parameter model, to test text generation.
00:01:49 High-Speed Image Generation: The platform supports image generation models like SDXL Turbo, which can quickly produce high-quality images from text prompts (e.g., "a dense rainforest with a hiking trail").
00:02:24 Text Embedding Functionality: Developers can use the embedding feature to rapidly convert text into numerical vector representations, which are essential for tasks like semantic search and retrieval-augmented generation (RAG) using vector databases.
00:02:49 Developer SDK and API Access: The platform is designed for developers, offering a Python SDK and a REST API. The interface provides sample code and direct links to full documentation for easy integration.
00:03:15 Integrated Tutorials and Code Samples: The tutorials section includes community-built guides and inline code samples that can be run directly in the browser to perform tasks like retrieving a list of all available models on the server.
00:03:50 Model Management and Rate Limits: A dedicated "Models Overview" section lists all hosted models and details their performance (tokens per second) and the user's specific rate limits (per minute, hour, and day) for each.
00:04:14 Usage Tracking and Transparent Pricing: The platform provides a clear breakdown of credit usage across different model types (text, image, embeddings) and offers free credits for new users to experiment. A dedicated pricing page lists the current costs for using each model.
00:04:47 Agentic AI Showcase: The video demonstrates the platform's capability to run AI agents with an example of a "Stockwatch" agent. This agent uses the 70B model to quickly analyze and display a stock's performance over the past year.
00:05:27 Support and Community Resources: The platform includes an FAQ section for guidance and provides a link to a community Discord server for developer support and collaboration.