Case of implementation of synchronization and video transcoding on Education domain projects.
We at SDH have already performed plenty of both complex and easy tasks. Sometimes projects have some similar, basic things. Features mentioned in the article are completely unlike others.
The customer set us the task to simplify and facilitate uploading educational information packages to the site. The package includes educational content for the teacher, which helps conduct classes with students, a methodical material.
The person responsible for creating the training content (let's call him Harry) has to upload it to the site. By then, the customer already had a Drupal-based admin panel, which was not very convenient. Harry creates a package of HTML pages. These HTML pages have a bright design, full of animations and videos.
Our task was to make that process easier for Harry to easily upload materials for teachers without using FTP and Drupal.
During the implementation of the project, we solved the following main problems:
Transcoding is a digital-to-digital conversion of one type of encoded data (video or audio) to another, often because the target device used to display the content requires a smaller file size.
It took us more than two months to write the entire project, including sections and functionality:
The following team engaged in the project:
The inconvenience of working with Drupal administration and voluminous content packages
One packet could weigh more than 2.5 GB. We immediately rejected the idea of loading it through a browser on the front-end. The reason was that it could take several hours to upload the package; simultaneously, you could not close the browser (and what if you accidentally closed it?). We proposed developing a synchronization with a third-party service installed on Harry's computer to avoid the risks of long uploads and errors. Therefore, uploading to the server does not depend on the browser but can occur in the background.
After the investigation stage, we focused on Dropbox and its external API.
The workflow was as follows:
Synchronization process
When synchronization started, a list of files was created in Dropbox to be copied to S3. This process looks as follows:
After the development stage, we started testing. The testing was supposed to find pitfalls that prevented synchronization—for example, unnecessary or damaged files. The case was considered to be positive if the message about successful synchronization appeared on the screen.
Video transcoding
The product we were developing is hosted on AWS, so Problem #2 was solved by employing its capabilities.
The content pack contained many video files. Video playback requirements were such that the video had to be played on all browsers and all devices. Previously, our customer had stored videos in packages in two formats: .mp4 and .ogg, which lengthened the time for package formation. We suggested transcoding video using AWS Elemental MediaConvert to streaming video:
Advantages
- when creating the package, Harry had to save the video in one .mp4 format;
- availability of playback in all browsers and on all devices.
How it works
The package contains many static files (HTML pages, CSS, and JS files) that reside on AWS S3.
When uploading a package to S3, a manifest with the number of video files in the package is added to the root. Each time a video file is created on AWS, a trigger is fired that increases the counter of created video files.
When the number of video files created in AWS reaches the number declared in the manifest, a job is started in which transcoding is triggered. Transcoding takes some time, depending on the length of the video. In case of success or vice versa, a completion request is sent to the endpoint.
Before uploading the package to S3, a js file is added, which initiates the player for streaming and the necessary changes in the root HTML (usually index.html).
When the player starts, a request is made to the backend; the response contains the desired playlist. It is essential to say that all root HTML should have a similar structure, classes, and element IDs, which simplifies the solution.
The testing phase was directly from the S3 production, meaning AWS did not have a sandbox. We pulled video files from our test server. The testing costs a lot of money for the customer because it is necessary to check how real video files were transcoded, and every second of transcoding has its price.
Technologies we used:
Python, Django, Celery, Boto3, BeautifulSoup
Read also: Top Education App Design Trends in 2023
The project team realized the customer's target, stopping Harry's headache. Harry has a user-friendly admin panel for uploading training content packages, including bulk files. Thanks to the synchronization with Dropbox, educational information uploads on the platform in just a few clicks, quickly, and with the possibility of background uploading.
The video transcoding function implemented with AWS Elemental MediaConvert allows users to avoid the problem of not playing files of different formats. Thus, video files played in streaming mode work correctly on all browsers and devices.
'Partnership that works for you' is our guiding principle. We take on complex tasks of custom software development. Over 80 companies have outsourced Software Development Hub to scale their engineering teams and build web applications.
Drop us a line, and we provide you with a qualified consultation.