Loading...

Cloud Storage Integration

Work with cloud storage providers in Python FFmpeg Video Streaming

View on GitHub v0.1
Cloud Storage

Integrate with S3, Google Cloud, Azure and custom providers

Table of Contents

Overview

You can open a file from a cloud (or other resources) and save packaged files to a cloud (clouds). These options are only valid for VOD (it does not support live streaming).

Amazon S3

Object storage through a web service interface

Google Cloud

RESTful online file storage web service

Azure Storage

Microsoft's cloud storage solution for modern data storage

Custom Cloud

Extend the Clouds class for your custom provider

Amazon S3

Amazon S3 or Amazon Simple Storage Service is a service offered by Amazon Web Services (AWS) that provides object storage through a web service interface. Amazon S3 uses the same scalable storage infrastructure that Amazon.com uses to run its global e-commerce network. Amazon S3 can be employed to store any type of object which allows for uses like storage for Internet applications, backup and recovery, disaster recovery, data archives, data lakes for analytics, and hybrid cloud storage. AWS launched Amazon S3 in the United States on March 14, 2006, then in Europe in November 2007. Learn more

Installation

First you need to install Amazon S3 dependency via pip:

pip install boto3

Usage

After installing the package, you can use the following code to open a file from the cloud and save packaged files to it.

from ffmpeg_streaming import S3, CloudManager

logging.basicConfig(filename='streaming.log', level=logging.NOTSET, format='[%(asctime)s] %(levelname)s: %(message)s')

# see https://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html to get Security Credentials
s3 = S3(aws_access_key_id='YOUR_KEY_ID', aws_secret_access_key='YOUR_KEY_SECRET', region_name='YOUR_REGION')

save_to_s3 = CloudManager().add(s3, bucket_name="bucket_name", folder="folder/subfolder")
# A filename can also be passed to the CloudManager class to change the filename(e.x. CloudManager(filename="hls.m3u8"))

video = ffmpeg_streaming.input(s3, bucket_name="bucket_name", key="video.mp4")
# You can also open a file from a local path
                                # video = ffmpeg_streaming.input("/var/media/video.mp4")

hls = video.hls(Formats.h264())
hls.auto_generate_representations()

hls.output(clouds=save_to_s3)
# you can also pass a local path to save a copy of files to your server
NOTE: For getting credentials, you need to have an AWS account or you can create one.
Read the "AWS SDK for Python" Document found here for more information.

Google Cloud Storage

Google Cloud Storage is a RESTful online file storage web service for storing and accessing data on Google Cloud Platform infrastructure. The service combines the performance and scalability of Google's cloud with advanced security and sharing capabilities. It is an Infrastructure as a Service (IaaS), comparable to Amazon S3 online storage service. Contrary to Google Drive and according to different service specifications, Google Cloud Storage appears to be more suitable for enterprises. Learn more

Installation

First you need to install Google Cloud Storage dependency via pip:

pip install google-cloud-storage

Usage

After installing the package, you can use the following code to open a file from the cloud and save packaged files to it.

from ffmpeg_streaming import GCS, CloudManager, Formats

logging.basicConfig(filename='streaming.log', level=logging.NOTSET, format='[%(asctime)s] %(levelname)s: %(message)s')

# Create a service account key from here: https://console.cloud.google.com/apis/credentials/serviceaccountkey
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "/var/keys/[YOUR-CREDENTIALS].json"
# if you are on windows, the path should be like: "C:\User\username\Desktop\[YOUR-CREDENTIALS].json"
gcs = GCS()

save_to_gcs = CloudManager().add(gcs, bucket_name="bucket_name", folder="folder/subfolder")
# A filename can also be passed to the CloudManager class to change the filename(e.x. CloudManager(filename="hls.m3u8"))

video = ffmpeg_streaming.input(gcs, bucket_name="bucket_name", object_name="video.mp4")
# you can also open a file from a local path
                                # video = ffmpeg_streaming.input("/var/media/video.mp4")

hls = video.hls(Formats.h264())
hls.auto_generate_representations()

hls.output(clouds=save_to_gcs)
# you can also pass a local path to save a copy of files to your server
NOTE: To get credentials, read the Cloud Storage Authentication found here or you can create it directly (Select the "Service account key" option).

Microsoft Azure Storage

Azure Storage is Microsoft's cloud storage solution for modern data storage scenarios. Azure Storage offers a massively scalable object store for data objects, a file system service for the cloud, a messaging store for reliable messaging, and a NoSQL store. Learn more

Installation

First you need to install Microsoft Azure Storage dependency via pip:

pip install azure-storage-blob

Usage

After installing the package, you can use the following code to open a file from the cloud and save packaged files to it.

from ffmpeg_streaming import MAS, CloudManager

logging.basicConfig(filename='streaming.log', level=logging.NOTSET, format='[%(asctime)s] %(levelname)s: %(message)s')

# see https://docs.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage to get credentials
mas = MAS(account_name='account_name', account_key='account_key')

save_to_mas = CloudManager().add(mas, container="container-name")
# A filename can also be passed to the CloudManager class to change the filename(e.x. CloudManager(filename="hls.m3u8"))

video = ffmpeg_streaming.input(mas, container="container-name", blob="video.mp4")
# you can also open a file from a local path
                                # video = ffmpeg_streaming.input("/var/media/video.mp4")

hls = video.hls(Formats.h264())
hls.auto_generate_representations()

hls.output(clouds=save_to_mas)
# you can also pass a local path to save a copy of files to your server
NOTE: To authenticate the service, see here.

Custom Cloud

You can upload your file to a custom cloud. You can create a custom cloud that is extended the "Clouds" class:

from ffmpeg_streaming import Clouds

class CustomCloud(Clouds):

    def upload_directory(self, directory, **options):
        # @TODO: upload a directory to a cloud
        pass

    def download(self, filename=None, **options):
        # @TODO: download a file to a local path
        pass

Usage

After creating your own cloud class, you can use the following code to open a file from your custom cloud and save packaged files to it.

custom = CustomCloud()

save_to_custom = CloudManager().add(custom, options="value")

video = ffmpeg_streaming.input(custom, option1="value1", option2="value2")
# you can also open a file from a local path
                                # video = ffmpeg_streaming.input("/var/media/video.mp4")

hls = video.hls(Formats.h264())
hls.auto_generate_representations()

hls.output(clouds=save_to_custom)
# you can also pass a local path to save a copy of files to your server

Save to multiple clouds

You can save your files to multiple clouds:

save_to_multiple_clouds = (
    CloudManager()
        .add(s3, bucket_name="bucket-name")
        .add(gcs, bucket_name="bucket-name")
        .add(mas, container="container-name")
        .add(custom, options="value")
)

hls.output(clouds=save_to_multiple_clouds)

A path can also be passed to save a copy of files on your local machine.

hls.output('/var/media/hls.m3u8', clouds=save_to_multiple_clouds)