The API offered by ‘Groq, Inc.’ for their Language Processing Unit (LPU) hardware - AI hardware specifically designed to accelerate performance for natural language processing (NLP) tasks.

Thanks to the Groq LPU API we have a programmatic way for developers to interact with the LPU and leverage its capabilities for their applications.

Build AI Assistants with memory, knowledge and tools.

  • You will need an API key from Groq to use the project - The models might be open, but you wont have full local control over them and send queries to a 3rd Party ❎

This is what we will get:

Groq Youtube Summaries

SelfHosting Groq Video Summaries

The phi-data repository contains several sample applications like this one, but our focus today will be at: phidata/cookbook/llms/groq/video_summary

It is very simple to get Youtube video summaries with Groq, we are going to do it systematically with Docker.

Really, Just Get Docker 🐋👇

You can install Docker for any PC, Mac, or Linux at home or in any cloud provider that you wish. It will just take a few moments. If you are on Linux, just:

apt-get update && sudo apt-get upgrade && curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh
#sudo apt install docker-compose -y

And install also Docker-compose with:

apt install docker-compose -y

When the process finishes, you can use it to self-host other services as well. You should see the versions with:

docker --version
docker-compose --version
#sudo systemctl status docker #and the status

We will use this Dockerfile to capture the App we are interested from the repository and bundle it:

FROM python:3.11

# Install git
RUN apt-get update && apt-get install -y git

# Set up the working directory
#WORKDIR /app

# Clone the repository
RUN git clone --depth=1 https://github.com/phidatahq/phidata && \
    cd phidata && \
    git sparse-checkout init && \
    git sparse-checkout set cookbook/llms/groq/video_summary && \
    git pull origin main

WORKDIR /phidata

# Install Python requirements
RUN pip install -r /phidata/cookbook/llms/groq/video_summary/requirements.txt

#RUN sed -i 's/numpy==1\.26\.4/numpy==1.24.4/; s/pandas==2\.2\.2/pandas==2.0.2/' requirements.txt


# Set the entrypoint to a bash shell
CMD ["/bin/bash"]

And now we just have to build our Docker Image with the Groq Youtube Summarizer App:

docker build -t phidata_yt_groq .

sudo docker-compose up -d
#docker-compose -f phidata_yt_groq_Docker-compose.yml up -d

docker exec -it phidata_yt_groq /bin/bash

Now, you are inside the built container.

Just execute the Streamlit app to test it:

#docker or podman run
docker run -d --name=phidata_yt_groq -p 8502:8501 -e GROQ_API_KEY=your_api_key_here \
phidata:yt_summary_groq streamlit run cookbook/llms/groq/video_summary/app.py

# podman run -d --name=phidata_yt_groq -p 8502:8501 -e GROQ_API_KEY=your_api_key_here \
# phidata:yt_summary_groq tail -f /dev/null
How to use Github Actions to build a multi-arch Container Image ⏬

If you are familiar with github actions, you will need the following workflow:

name: CI/CD Build MultiArch

on:
  push:
    branches:
      - main

jobs:
  build-and-push:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout repository
      uses: actions/checkout@v2

    - name: Set up QEMU
      uses: docker/setup-qemu-action@v1

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v1

    - name: Login to GitHub Container Registry
      uses: docker/login-action@v1
      with:
        registry: ghcr.io
        username: ${{ github.actor }}
        password: ${{ secrets.CICD_TOKEN_YTGroq }}

    - name: Set lowercase owner name
      run: |
                echo "OWNER_LC=${OWNER,,}" >> $GITHUB_ENV
      env:
        OWNER: '${{ github.repository_owner }}'

    - name: Build and push Docker image
      uses: docker/build-push-action@v2
      with:
        context: .
        push: true
        platforms: linux/amd64,linux/arm64
        tags: |
                    ghcr.io/${{ env.OWNER_LC }}/phidata:yt-groq

It uses QEMU together with docker buildx command to build x86 and ARM64 container images for you.

And once you are done, deploy it as a Docker Compose Stack:

version: '3.8'

services:
  phidata_service:
    image: phidata_yt_groq #ghcr.io/jalcocert/phidata:yt-groq
    container_name: phidata_yt_groq
    ports:
      - "8501:8501"    
    environment:
      - GROQ_API_KEY=your_api_key_here #your_api_key_here 😝 
    command: streamlit run cookbook/llms/groq/video_summary/app.py 
    #command: tail -f /dev/null # Keep the container running
#     networks:
#       - cloudflare_tunnel
#       - nginx_default
            
# networks:
#   cloudflare_tunnel:
#     external: true
#   nginx_default:
#     external: true

For Production deployment, you can use NGINX or Cloudflare Tunnels to get HTTPs

How to SelfHost the Streamlit AI App with Cloudflare ⏬
  1. Visit - https://one.dash.cloudflare.com
  2. Hit Tunnels (Network section)
  3. Create a new Tunnel

Streamlit with Cloudflare Tunnels

And make sure to place the name of the container and its internal port:

Streamlit with Cloudflare Tunnels


Conclusion

Now we have our Streamlit UI at: localhost:8501

Feel free to ask for Summaries about Youtube Videos with Groq:

Groq Youtube Summaries Example

Similar AI Projects 👇

Adding Simple Streamlit Auth

We can use this simple package: https://pypi.org/project/streamlit-authenticator/

A secure authentication module to validate user credentials in a Streamlit application.

import streamlit as st
from Z_Functions import Auth_functions as af


def main():
    if af.login():
        # Streamlit UI setup
        st.title("Portfolio Dividend Visualization")

if __name__ == "__main__":
    main()        

It is a referencing a Auth_functions.py in another file:

Define Auth_functions.py in a separate file with ⏬
import streamlit_authenticator as stauth

# Authentication function #https://github.com/naashonomics/pandas_templates/blob/master/login.py
def login():
    names = ['User Nick 1 🐷', 'User Nick 2']
    usernames = ['User 1', 'User 2']
    passwords = ['SomePassForUser1', 'anotherpassword']

    hashed_passwords = stauth.Hasher(passwords).generate()
    authenticator = stauth.Authenticate(names, usernames, hashed_passwords,
                                        'some_cookie_name', 'some_signature_key', cookie_expiry_days=1)
    name, authentication_status, username = authenticator.login('Login', 'main')
    
    if authentication_status:
        authenticator.logout('Logout', 'main')
        st.write(f'Welcome *{name}*')
        return True
    elif authentication_status == False:
        st.error('Username/password is incorrect')
    elif authentication_status == None:
        st.warning('Please enter your username (🐷) and password (💰)')
    return False

FAQ

How to use Groq API Step by Step with Python ⏬

Thanks to TirendazAcademy, we have a step by step guide to know how to make queries to Groq API via Python:

!pip install -q -U langchain==0.2.6 langchain_core==0.2.10 langchain_groq==0.1.5 gradio==4.37.1
from google.colab import userdata
groq_api_key = 'your_groq_API'

from langchain_groq import ChatGroq

chat = ChatGroq(
    api_key = groq_api_key,
    model_name = "mixtral-8x7b-32768"
)

from langchain_core.prompts import ChatPromptTemplate

system = "You are a helpful assistant."
human = "{text}"

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system), ("human", human)
    ]
)

from langchain_core.output_parsers import StrOutputParser
chain = prompt | chat | StrOutputParser()

chain.invoke(
    {"text":"Why is the sky blue?"}
)

Groq vs Others

  • Ollama - A free and open-source AI service that features local models, allowing users to host their own machine learning models. This service is ideal for users who prefer to keep their data on their own servers.

  • Text Generation Web UI - This is a free and open-source AI service that uses local models to generate text. It’s a great tool for content creators and writers who need a quick and easy way to generate text.

  • Mistral - An AI service that specializes in creating personalized experiences for users. It uses machine learning algorithms to understand user behavior and preferences.

  • Grok (Twitter) - An AI service developed by Twitter for data analysis and pattern recognition. It’s primarily used for analyzing social media data to provide insights into user behavior.

  • Gemini (Google) - An AI service from Google that provides APIs for machine learning and data analysis. You can access the documentation here

  • Vertex AI (Google) - Another AI service from Google, Vertex AI provides tools for data scientists and developers to build, deploy, and scale AI models.

  • AWS Bedrock - This is Amazon’s foundational system for AI and ML services. It provides a wide range of tools and services for building, training, and deploying machine learning models.

  • Anthropic (Claude) - A research-oriented AI service developed by Anthropic. It aims to build models that understand and respect human values. You can access their API here and manage your settings and keys on their console.

Ways to Secure a Streamlit App

We have already seen a simple way with the Streamlit Auth Package.

But what if we need something more robust?

How to secure the Access for your AI Apps

  • F/OSS Apps to manage Application access Management:
    • Authentik
    • LogTo:

https://docs.logto.io/docs/recipes/protected-app/ https://docs.logto.io/docs/recipes/manage-users/admin-console/ https://docs.logto.io/docs/recipes/webhooks/ –» https://cloud.logto.io/yourtenantid/webhooks

It provides an email to authenticate users.

  • Create a Protected App: Add authentication with simplicity and epic speed

https://www.youtube.com/watch?v=ZBrXW3iZyKY

Build your app’s authentication in clicks, no code required

How to install AI easily

Some F/OSS Projects to help us get started with AI:

FREE Vector Stores for AI Projects