Model¶

Instances of the Model class represent machine learning models managed by or integrated with BerryDB. You typically obtain Model objects by using the ModelRepo.get() method.

While BerryDB supports different model providers (like Hugging Face, Vertex AI, Custom Models), the SDK provides a unified Model interface. The availability of certain methods like deploy, request, or shutdown may depend on the specific provider and the model’s configuration, as detailed in each method’s documentation.

Model Methods¶

Model.status()¶

Retrieves the current status of the model from BerryDB.

This method queries BerryDB to get the operational status of the deployed or requested model.

Returns:

str: A string representing the current status of the model (e.g., “AVAILABLE”, “REQUESTED”, “UNAVAILABLE” …).

Example:

from berrydb import BerryDB, ModelProvider

# Assuming you have a Model instance, e.g., fetched from the ModelRepo
repo = BerryDB.model_repo("BERRYDB_API_KEY")
my_model = repo.get(provider=ModelProvider.CUSTOM_MODEL, model_name="my-deployed-model")

# Check the model's status
current_status = my_model.status()
print(f"Model '{my_model.config.name}' status: {current_status}")

# You might loop and check status until it's 'DEPLOYED'
# while my_model.status() != "AVAILABLE":
#     print("Model is not yet available, waiting...")
#     time.sleep(10) # Wait for 10 seconds before checking again
# print("Model is now available!")

Model.predict(input_data, annotation_config=None)¶

Retrieves a prediction from the model hosted on BerryDB.

This method sends input data to the deployed model and returns the prediction generated by the model. The structure and type of the input and output data depend on the specific model being used.

Parameters:

input_data (any): The input data required by the model to generate a prediction. The format and type of this data are specific to the model’s requirements.
annotation_config (str): Annotation config name if the model’s input and output needs to be transformed.

Returns:

object: The prediction result returned by the model. The structure and type of this object depend on the model’s output format.

Example:

from berrydb import BerryDB, ModelProvider

# Assuming you have a Model instance, e.g., fetched from the ModelRepo
repo = BerryDB.model_repo("BERRYDB_API_KEY")
my_model = repo.get(provider=ModelProvider.HUGGING_FACE_MODEL, model_name="my-sentiment-analyzer")

# Example input data (this will vary based on your model)
text_to_predict = "This movie is absolutely fantastic!"

# Get a prediction from the model
prediction_result = my_model.predict(text_to_predict)
print(f"Prediction for '{text_to_predict}': {prediction_result}")

# Another example with structured input (if your model expects it)
# structured_input = {"text": "The service was terrible.", "language": "en"}
# prediction_result = my_model.predict(structured_input)
# print(f"Prediction: {prediction_result}")

HuggingFaceModel Methods¶

HuggingFaceModel.request()¶

Requests the setup or deployment of a Hugging Face model in BerryDB.

This method is used for models obtained from the Hugging Face provider. If the model’s current status is ‘UNAVAILABLE’, calling this method will initiate the process to make it available for predictions. If the model has already been requested or is in a state other than ‘UNAVAILABLE’, this method may indicate that the action cannot be performed.

Returns:

dict: A dictionary containing the response from BerryDB, typically indicating the success or status of the request.

Raises:

Exception: If BerryDB returns an error, for example, if the model is not in a state where it can be requested (e.g., already ‘AWAITING’ or ‘REQUESTED’).

Example:

from berrydb import BerryDB, ModelProvider

berrydb_api_key = "BERRYDB_API_KEY"
repo = BerryDB.model_repo(berrydb_api_key)

try:
    # Get a Hugging Face model that might be 'UNAVAILABLE'
    hf_model = repo.get(
        provider=ModelProvider.HUGGING_FACE_MODEL,
        model_name="my-hf-model-to-request"
    )

    if hf_model and hf_model.config.status() == "UNAVAILABLE":
        print(f"Model '{hf_model.config.name}' is unavailable. Requesting deployment...")
        request_response = hf_model.request()
        print(f"Model request initiated: {request_response}")
        # You might want to check its status periodically after this
        # print(f"New status: {hf_model.status()}")
    elif hf_model:
        print(f"Model '{hf_model.config.name}' status is '{hf_model.config._hf_status}'. No request needed or possible.")
    else:
        print("Model not found.")

except Exception as e:
    print(f"An error occurred: {e}")

VertexAIModel Methods¶

VertexAIModel.deploy(machine_type=None, min_replicas=None, max_replicas=None)¶

Deploys a VertexAI model hosted by BerryDB.

This method initiates the deployment process for a VertexAI model that has been requested. It allows you to specify the machine type and scaling configuration for the deployment.

This method is available for Model instances obtained for the ModelProvider.VERTEX_AI_MODEL providers.

Parameters:

machine_type (str, optional): The type of machine to use for the model replicas (e.g., “n1-standard-4”). Defaults to a standard machine type if not specified.
min_replicas (int, optional): The minimum number of model replicas to maintain. Defaults to 1. Must be between 1 and 3.
max_replicas (int, optional): The maximum number of model replicas to scale up to. Defaults to 1. Must be between 1 and 3.

Returns:

dict: A dictionary containing the response from BerryDB, typically confirming the deployment request and providing details about the initiated process.

Raises:

ValueError: If min_replicas or max_replicas are outside the allowed range (1-3) or if min_replicas is greater than max_replicas.

Example:

from berrydb import BerryDB, ModelProvider, ModelConfig

# Initialize BerryDB (if not already done)
# BerryDB.init("your-berrydb-host")

berrydb_api_key = "BERRYDB_API_KEY"
repo = BerryDB.model_repo(berrydb_api_key)

# Assuming you have previously requested a VertexAI model and now want to deploy it.
# You can retrieve the model instance using repo.get()
try:
    vertexai_model_to_deploy = repo.get(
        provider=ModelProvider.VERTEX_AI_MODEL,
        model_name="my-image-classifier" # Name of your saved VertexAI model
    )

    if vertexai_model_to_deploy:
        print(f"Found VertexAI model: {vertexai_model_to_deploy.config.name}")

        # Deploy the model with specific scaling settings
        print("Requesting deployment...")
        deployment_response = vertexai_model_to_deploy.deploy(
            machine_type="n1-standard-4",
            min_replicas=1,
            max_replicas=2
        )
        print(f"Deployment initiated: {deployment_response}")

        # You can check the status periodically using vertexai_model_to_deploy.status()
        # until it shows as 'AVAILABLE'.

    else:
        print("VertexAI model not found.")

except ValueError as ve:
    print(f"Deployment configuration error: {ve}")
except Exception as e:
    print(f"An error occurred during deployment: {e}")

CustomModel Methods¶

CustomModel.deploy(machine_type=None, min_replicas=None, max_replicas=None)¶

Deploys a Custom model hosted by BerryDB.

This method initiates the deployment process for a Custom model that has been saved to BerryDB (i.e., not self-hosted). It allows you to specify the machine type and scaling configuration for the deployment.

This method is available for Model instances obtained for the ModelProvider.CUSTOM_MODEL (when not self-hosted).

Parameters:

machine_type (str, optional): The type of machine to use for the model replicas (e.g., “n1-standard-4”). Defaults to a standard machine type if not specified.
min_replicas (int, optional): The minimum number of model replicas to maintain. Defaults to 1. Must be between 1 and 3.
max_replicas (int, optional): The maximum number of model replicas to scale up to. Defaults to 1. Must be between 1 and 3.

Returns:

dict: A dictionary containing the response from BerryDB, typically confirming the deployment request and providing details about the initiated process.

Raises:

ValueError: If min_replicas or max_replicas are outside the allowed range (1-3) or if min_replicas is greater than max_replicas.

Example:

from berrydb import BerryDB, ModelProvider, ModelConfig

# Initialize BerryDB (if not already done)
# BerryDB.init("your-berrydb-host")

berrydb_api_key = "BERRYDB_API_KEY"
repo = BerryDB.model_repo(berrydb_api_key)

# Assuming you have previously saved a non-self-hosted Custom model
# and now want to deploy it.
# You can retrieve the model instance using repo.get()
try:
    custom_model_to_deploy = repo.get(
        provider=ModelProvider.CUSTOM_MODEL,
        model_name="my-image-classifier" # Name of your saved custom model
    )

    if custom_model_to_deploy:
        print(f"Found Custom model: {custom_model_to_deploy.config.name}")

        # Deploy the model with specific scaling settings
        print("Requesting deployment...")
        deployment_response = custom_model_to_deploy.deploy(
            machine_type="n1-standard-4",
            min_replicas=1,
            max_replicas=2
        )
        print(f"Deployment initiated: {deployment_response}")

        # You can check the status periodically using custom_model_to_deploy.status()
        # until it shows as 'AVAILABLE'.

    else:
        print("Custom model not found.")

except ValueError as ve:
    print(f"Deployment configuration error: {ve}")
except Exception as e:
    print(f"An error occurred during deployment: {e}")

CustomModel.shutdown()¶

Shuts down a deployed Custom model hosted by BerryDB.

This method terminates the running instances (replicas) of a Custom model that was previously deployed on BerryDB. After calling this method, the model will no longer be available for predictions until it is deployed again.

This method is available for Model instances obtained for the ModelProvider.CUSTOM_MODEL provider (specifically, non-self-hosted models that have been deployed).

Returns:

dict: A dictionary containing the response from BerryDB, typically confirming the shutdown request.

Example:

from berrydb import BerryDB, ModelProvider

# Initialize BerryDB (if not already done)
# BerryDB.init("your-berrydb-host")

berrydb_api_key = "BERRYDB_API_KEY"
repo = BerryDB.model_repo(berrydb_api_key)

# Assuming you have a deployed non-self-hosted Custom model
# and now want to shut it down.
# You can retrieve the model instance using repo.get()
try:
    custom_model_to_shutdown = repo.get(
        provider=ModelProvider.CUSTOM_MODEL,
        model_name="my-image-classifier" # Name of your deployed custom model
    )

    if custom_model_to_shutdown:
        print(f"Found Custom model: {custom_model_to_shutdown.config.name}")

        # Shut down the model
        print("Requesting shutdown...")
        shutdown_response = custom_model_to_shutdown.shutdown()
        print(f"Shutdown initiated: {shutdown_response}")

        # You can check the status periodically using custom_model_to_shutdown.status()
        # until it shows as 'UNAVAILABLE'.

    else:
        print("Custom model not found.")

except Exception as e:
    print(f"An error occurred during shutdown: {e}")