BerryDB Python SDK

Welcome to the documentation of BerryDB Python SDK.

Introduction 👋

The BerryDB Python SDK offers a complete toolkit for developers and data scientists to interact with the BerryDB data platform. It’s designed to streamline the entire machine learning (ML) workflow, from managing datasets and annotations to handling model development and evaluation. 🚀

---

Core Components 🏗️

These modules provide foundational access to your data and annotation projects.

• BerryDB and Database: These are the primary entry points for connecting to and managing your BerryDB databases, allowing for fundamental data access and manipulation. 💾

• Annotation Project and Annotations Config: Central to creating and configuring data labeling tasks, these modules help define the structure and rules for annotating datasets, crucial for building high-quality training data. 🏷️

---

Model Management 🤖

This suite of modules offers a robust framework for versioning, storing, and configuring your machine learning models.

• Model Repo, Model Config, and Model: Together, these components facilitate the complete lifecycle of a model within the BerryDB ecosystem, ensuring models are reproducible and simplifying the path to deployment. 🔄

---

Evaluation and Settings 📊

These modules focus on assessing model performance and customizing the SDK’s behavior.

• Evaluator: An essential tool for a critical step in the ML lifecycle, providing functionality to run evaluations, compare model performance, and analyze results. ✅

• Settings: Allows for the customization of the SDK’s behavior, enabling you to tailor various configurations to your specific project requirements. ⚙️

---

---

Table of Contents

Contents

• Installation
• BerryDB
  • BerryDB.init()
  • BerryDB.connect()
  • BerryDB.databases()
  • BerryDB.create_schema()
  • BerryDB.create_database()
  • BerryDB.delete_database()
  • BerryDB.create_annotation_project()
  • BerryDB.model_repo()
• Database
  • Database.settings()
  • Database.enable_fts()
  • Database.database_name()
  • Database.get_all_documents()
  • Database.get_all_documents_with_col_filter()
  • Database.get_document_by_object_id()
  • Database.query()
  • Database.upsert()
  • Database.ingest_pdf()
  • Database.embed()
  • Database.chat()
  • Database.chat_for_eval()
  • Database.similarity_search()
  • Database.evaluator()
  • Database.ner()
  • Database.text_classification()
• Annotation Project
  • AnnotationProject Methods
    • AnnotationProject.setup_label_config()
    • AnnotationProject.populate()
    • AnnotationProject.connect_to_ml()
    • AnnotationProject.retrieve_prediction()
    • AnnotationProject.create_prediction()
    • AnnotationProject.create_annotation()
    • AnnotationProject.attach_annotations_config()
    • AnnotationProject.get_task_data()
• Annotations Config
  • AnnotationsConfig.builder()
  • AnnotationsConfig.save()
  • AnnotationsConfig.get_config_as_builder()
  • Annotations Config Builder Methods
    • Builder.name()
    • Builder.input_transform_expression()
    • Builder.output_transform_expression()
    • Builder.additional_transform_expressions()
    • Builder.llm_provider()
    • Builder.llm_model()
    • Builder.prompt()
    • Builder.build()
• Model Repo
  • ModelRepo.vertexai_models_for_request()
  • ModelRepo.get()
  • ModelRepo.get_by_name()
  • ModelRepo.save()
  • ModelRepo.request()
• Model Config
  • ModelConfig Static Methods
    • Fetching Available Model Categories
  • ModelConfig (Builder Entry Points)
    • ModelConfig.huggingface_builder()
    • ModelConfig.vertexai_builder()
    • ModelConfig.berrydb_builder()
  • ModelConfig.Builder Methods
    • Hugging Face Specific Builder Methods
    • Vertex AI Specific Builder Methods
    • Custom Model Specific Builder Methods
    • BerryDB Model Specific Builder Methods
• Model
  • Model Methods
    • Model.status()
    • Model.predict()
  • HuggingFaceModel Methods
    • HuggingFaceModel.request()
  • VertexAIModel Methods
    • VertexAIModel.deploy()
  • CustomModel Methods
    • CustomModel.deploy()
    • CustomModel.shutdown()
• Settings
  • Builder.retrieval_settings()
  • Builder.transform_settings()
  • Builder.embed_settings()
  • Builder.chat_settings()
• Evaluator
  • BerryDBRAGEvaluator.eval()