Skip to content

Installation

llamatelemetry v0.1.0 is designed for Kaggle dual T4 notebooks with pre-built CUDA binaries optimized for Tesla T4 GPUs.

Requirements

  • Platform: Kaggle Notebooks
  • GPUs: 2× Tesla T4 (15GB VRAM each)
  • CUDA: 12.x (pre-installed on Kaggle)
  • Python: 3.11+
  • Internet: Required for initial setup

Minimum

  • GPUs: 1× Tesla T4 (15GB VRAM)
  • CUDA: 12.0+
  • Python: 3.10+

Quick Install (Kaggle)

Install llamatelemetry directly from GitHub:

!pip install -q --no-cache-dir --force-reinstall \
    git+https://github.com/llamatelemetry/llamatelemetry.git@v0.1.0

What Happens During Installation

  1. Python package installation - Installs llamatelemetry Python SDK
  2. Binary download - Downloads pre-built CUDA binaries (~961 MB) on first import
  3. GPU detection - Automatically detects Tesla T4 GPUs
  4. Binary verification - Verifies SHA256 checksums
  5. FlashAttention setup - Configures FlashAttention v2 support

Verify Installation

After installation, verify everything is working:

import llamatelemetry
print(f"Version: {llamatelemetry.__version__}")

from llamatelemetry import check_cuda_available, get_cuda_device_info
from llamatelemetry.api.multigpu import gpu_count

print(f"CUDA Available: {check_cuda_available()}")
print(f"GPU Count: {gpu_count()}")

cuda_info = get_cuda_device_info()
if cuda_info:
    print(f"CUDA Version: {cuda_info.get('cuda_version', 'N/A')}")

Expected output:

Version: 0.1.0
CUDA Available: True
GPU Count: 2
CUDA Version: 12.5

Binary Download Details

On first import, llamatelemetry automatically downloads optimized binaries:

  • Source: HuggingFace Hub (waqasm86/llamatelemetry-binaries)
  • Size: ~961 MB
  • Features: FlashAttention + Tensor Cores + Multi-GPU tensor-split
  • Cache location: /root/.cache/llamatelemetry/ (Linux/Kaggle)
  • Verification: SHA256 checksum verification

Manual Binary Download

If automatic download fails, you can manually download:

from llamatelemetry.binaries.download import download_and_install_binaries

download_and_install_binaries(
    version="0.1.0",
    platform="kaggle-t4x2"
)

Kaggle Setup

Enable Dual T4 GPUs

  1. Open your Kaggle notebook
  2. Click Settings (gear icon, right sidebar)
  3. Under Accelerator, select GPU T4 x2
  4. Click Save
  5. Restart the notebook session

Enable Internet

  1. In notebook settings, ensure Internet is ON
  2. Required for downloading models and binaries

Dependencies

llamatelemetry automatically installs these dependencies:

Core Dependencies

  • opentelemetry-api>=1.39.0 - OpenTelemetry API
  • opentelemetry-sdk>=1.39.0 - OpenTelemetry SDK
  • opentelemetry-exporter-otlp>=1.39.0 - OTLP exporter
  • opentelemetry-semantic-conventions>=0.51b0 - Semantic conventions

Visualization Dependencies

  • graphistry>=0.34.0 - GPU-accelerated graph visualization
  • cudf-cu12>=24.0.0 - GPU dataframes (RAPIDS)
  • plotly>=5.18.0 - Interactive plotting

Inference Dependencies

  • huggingface-hub>=0.25.0 - Model download
  • pynvml>=12.0.0 - GPU monitoring

Troubleshooting

Binary Download Fails

If binary download fails, try:

# Clear cache and retry
import shutil
shutil.rmtree("/root/.cache/llamatelemetry/", ignore_errors=True)

# Reinstall
!pip install -q --no-cache-dir --force-reinstall \
    git+https://github.com/llamatelemetry/llamatelemetry.git@v0.1.0

CUDA Not Available

Verify CUDA installation:

!nvcc --version
!nvidia-smi

If CUDA is not available:

  1. Ensure GPU is enabled in Kaggle settings
  2. Restart notebook session
  3. Check that accelerator is set to "GPU T4 x2"

ImportError

If you see import errors, ensure all dependencies are installed:

!pip install --upgrade \
    opentelemetry-api \
    opentelemetry-sdk \
    opentelemetry-exporter-otlp \
    graphistry \
    plotly \
    huggingface-hub

Version Conflicts

llamatelemetry requires specific versions of OpenTelemetry packages. If you encounter version conflicts:

# Force reinstall with no cache
!pip install -q --no-cache-dir --force-reinstall \
    git+https://github.com/llamatelemetry/llamatelemetry.git@v0.1.0

Alternative Installation Methods

Install from Source

For development or custom builds:

git clone https://github.com/llamatelemetry/llamatelemetry.git
cd llamatelemetry
pip install -e .

See Build from Source for details.

Install Specific Version

Install a specific release version:

!pip install git+https://github.com/llamatelemetry/llamatelemetry.git@v0.1.0

Next Steps

After installation:

  1. Quick Start Guide - Get running in 5 minutes
  2. First Steps - Understand basic concepts
  3. Tutorial 01: Quick Start - First hands-on tutorial

Need Help?