Snowflake & Vertex AI Foundational LLMs
Snowflake Cortex gives you instant access to industry-leading large language models via a native & managed service running within the security perimeter of your Snowflake account, but have you ever wondered about how to leverage Google Vertex AI LLMs such as Gemini or PaLM2 on your Snowflake data? By design, these foundation models are deployed once and managed centrally by Google which means you’ll need to use Snowflake External Access to integrate with the Vertex APIs.
This pattern is very similar to other Snowflake Quickstarts that demonstrate how to integrate OpenAI and AWS Bedrock with Snowflake, and is aligned to Snowflake’s strategy of providing multiple options for building LLM-powered applications and securely bringing generative AI to governed Snowflake data:
The code and setup instructions are available from my github repo here and contains 4 files:
Customer_Reviews.csv: A csv file containing fictitious customer reviews for a retailer (you can upload this to a table named DEMOS.VERTEX.REVIEWS using Snowsight).
Vertex_Demo.sql: A sample script that demonstrates how to create a Stored Procedure/UDF that calls the Vertex API endpoint for Text (text-bison) foundational model which is ideal for tasks that can be completed with one API response: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text. The script includes the code for creating a Snowflake Security Integration, Secret, Network Rule, and External Access Integration used by the Stored Procedure.
VertexAI_Text_Gen_SiS.py: A ‘test harness’ UI for calling the Stored Procedure created in Step 2. This is a modified version of a standalone Streamlit application available from Google’s Github Repo here.
Customer_Review_Analyser_SiS.py: An example Streamlit in Snowflake application that provides a user with the ability to analyse/filter customer reviews stored in a Snowflake table and uses Vertex AI to summarize the review, perform sentiment analysis, and extract product information.
There’s also a 6 minute video showing how to setup the integration below: