BentoML

Model Serving Made Easy

Sep 23, 2021 | - views

BentoML is a flexible, high-performance framework for serving, managing, and deploying machine learning models. BentoML bridges the gap between Data Science and DevOps. By providing a standard interface for describing a prediction service, BentoML abstracts away how to run model inference efficiently and how model serving workloads can integrate with cloud infrastructures. See how it works!

pandas==1.3.3
scikit-learn==0.24.2
BentoML==0.13.1

Create app.py:

import bentoml
from bentoml.adapters import DataframeInput
from bentoml.frameworks.sklearn import SklearnModelArtifact
from sklearn.datasets import load_boston
from sklearn.ensemble import RandomForestRegressor


@bentoml.env(infer_pip_packages=True)
@bentoml.artifacts([SklearnModelArtifact('model')])
class SklearnModelService(bentoml.BentoService):
    @bentoml.api(input=DataframeInput(), batch=True)
    def predict(self, df):
        result = self.artifacts.model.predict(df)
        return result


def main():
    x, y = load_boston(return_X_y=True)
    estimator = RandomForestRegressor()
    estimator.fit(x, y)

    service = SklearnModelService()
    service.pack('model', estimator)
    service.save()


if __name__ == '__main__':
    main()
[2021-09-23 10:21:38,391] INFO - BentoService bundle 'SklearnModelService:20210923102137_1C3285' saved to: /home/mark/bentoml/repository/SklearnModelService/20210923102137_1C3285

Go to /home/mandreev/bentoml/repository/SklearnModelService/20210923102137_1C3285 and build docker

docker run -p 5000:5000 $(docker build -q /home/mark/bentoml/repository/SklearnModelService/20210923102137_1C3285)

Try to fetch predict with REST:

import requests

N = 13

if __name__ == '__main__':
    response = requests.post("http://127.0.0.1:5000/predict", json=[[1] * N])
    print(response.text)
[44.21300000000002]
Swagger ui available at http://127.0.0.1:5000/predict
Related articles: