reInvent is at its middle with full-throttle happening in the past 2 days with announcements made by Peter DeSantis and Matt Garman. More announcements have been made by the one and only Dr. Swami Sivasubramanian who is the VP of AI in AWS.
Most of these announcements are around the Amazon Bedrock product which is the easiest way to build and and scale GenAI applications and Foundational models on AWS. Features that will continue to improve user and customer experience on how they integrate it with application to generate text, image and video content which some of the most sophisticated Models out there.
Amazon Bedrock is a consolidation of so many standard and vast Foundational Models from industry leaders including Claude Mistral, Meta and the all new Amazon Nova.
Without Further Ado… these are the announcements made today by Swami
Swami’s Announcements in re:Invent 2024
Amazon Sagemaker Hyperpod task governance: Amazon SageMaker HyperPod task governance is now generally available, offering centralized management of GPU and Trainium resources for generative AI tasks like training, fine-tuning, and inference. This innovation addresses challenges in resource allocation, helping organizations avoid inefficiencies and delays caused by underutilized or stalled compute resources.
With HyperPod, administrators can set quotas based on project budgets and priorities, while tasks are automatically scheduled and executed within these limits.
The system dynamically reallocates resources by pausing and resuming low-priority tasks to prioritize critical ones, ensuring efficient use of compute capacity. Idle resources can also be reassigned to expedite waiting tasks across teams.
Users can monitor task queues, adjust priorities, and track compute usage, while administrators can audit and optimize resource allocations to control costs and improve availability. HyperPod enhances resource efficiency and accelerates the delivery of AI innovations by promoting timely completion of high-priority projects.
New AI models coming to the Amazon Bedrock Family
Amazon Bedrock Marketplace: Explore, evaluate, and leverage a wide range of over 100 emerging and specialized foundation models, supported by Amazon Bedrock’s robust tools, security, and governance.
Amazon Bedrock supports prompt caching (preview): Amazon Bedrock now supports caching frequently used context in prompts across model invocations, ideal for applications like document Q&A and coding assistants. Cached context is retained for up to 5 minutes after access, enabling cost reductions of up to 90% and latency improvements of up to 85% for supported models. These features simplify achieving lower latency and cost-efficient performance in your applications.
Amazon Bedrock Intelligent prompt routing: Amazon Bedrock now supports Intelligent Prompt Routing, enabling the use of foundation models from the same family to balance quality and cost.
For example, it can route requests between models like Claude 3.5 Sonnet and Claude 3 Haiku based on prompt complexity. This feature reduces costs by up to 30% while maintaining accuracy, making it ideal for applications like customer service.
Amazon Bedrock Knowledge base support for structured data retrieval: Amazon Bedrock Knowledge Bases enable natural language querying of data warehouses and data lakes, integrating critical enterprise data to enhance response accuracy.
As a fully managed RAG solution, it simplifies breaking down data silos and accelerates AI application development from weeks to days.
Businesses can use this capability to streamline workflows, such as automating claims adjudication for insurers, analyzing TV content for smart advertising, summarizing video assets for media production, or processing complex financial documents to identify relationships using GraphRAG. These solutions combine structured data retrieval and knowledge bases to enhance productivity and decision-making across industries.
Amazon Bedrock Knowledge Bases now supports GraphRAG (preview): Amazon Bedrock now offers fully-managed GraphRAG capabilities, combining Retrieval-Augmented Generation (RAG) with graph techniques to deliver more accurate and comprehensive responses for generative AI applications.
Amazon Bedrock Data Automation (preview): Amazon Bedrock Data Automation simplifies generating insights from unstructured, multimodal content like documents, images, audio, and videos. It supports tasks like intelligent document processing, media analysis, video summaries, and inappropriate content detection, with customizable outputs to meet specific business needs. This fully managed capability can operate standalone or as a parser for knowledge base setup in RAG workflows.
Amazon Bedrock Guardrails multimodal toxicity detection (preview): Amazon Bedrock Guardrails now includes multimodal toxicity detection, allowing the detection and filtering of harmful image content alongside text. This capability enhances user experiences and supports responsible AI by safeguarding generative AI applications against undesirable outputs.
With configurable policies for filtering topics, harmful content, PII redaction, and contextual checks, Guardrails ensures flexible compliance with specific use cases. The new image content filter detects categories like hate, violence, and explicit material, with adjustable thresholds to suit application needs. Compatible with all foundation models and custom fine-tuned models, this feature provides consistent protection across text and image modalities, simplifying the creation of secure AI solutions.
Amazon Q developer now available in Amazon Sagemaker Canvas
Amazon Kendra Gen AI Index
Scenario Analysis Capability with Amazon Q on Amazon Quicksight: Amazon QuickSight’s new Amazon Q capability helps users perform complex scenario analyses quickly and efficiently with AI assistance.
It guides business users through in-depth data analysis using natural language prompts, suggesting approaches, automatically analyzing data, and providing summarized findings and actions.
This feature speeds up scenario analysis by up to 10x compared to traditional spreadsheet methods, eliminating the need for manual, error-prone work.
For example, a franchisee could use Amazon Q to analyze performance differences between store locations, with the system suggesting relevant analysis steps, generating visualizations, and offering actionable insights. Users can explore multiple analysis paths and adjust as needed, enhancing decision-making and productivity.
