How video hosting works: explain how content is segmented and encrypted

How video hosting works: explain how content is segmented and encrypted

People are watching more and more videos: worldwide, the average daily use of video is already 84 minutes, and in 2020, more than 3 billion Internet users watched streaming or downloading videos at least once a month.

How did the video come about?

We learn what we see better than what we read or hear. An important development in visual communication was the appearance of a photograph in the 20s of the XlX century. It is well known that the events have evolved more rapidly: in 1895, the Lumiere brothers presented to the world a synematograph, a device for recording and replaying silent films. The sound film was short-lived, and in 1927 the first successful sound film, "Jaz singer", was presented, even though the recording technology had appeared in 1919.

The next stage in the development of the video format was the appearance of television, when the video was switched from cinemas to a daily broadcast format in flats. In fact, television appeared at the end of the 1930s, but it was really popular in the 1950s. Around the same time, the Kinescopes became relevant, a variety of electronic ray tubes used in television broadcasting. They owe it to Russian engineer Vladimir Zvorykin, who emigrated in the United States at the beginning of the twentieth century.

With the development of television broadcasting, television content became more complex, more expensive, more time-consuming and "unfree" — all broadcasts were clearly regulated and dependent on television ratings. The Internet completely changed the video world. In 1993, the Silver Tyre Damage group held the first live online concert. This impressed the entire world: a year later, a separate show was released, in which the hosts tried to understand the concept of the World Wide Web. The launch of ActiveMovie in 1995 allowed users to broadcast the video content, while the P2P Napster file network, which appeared in 1999, facilitated the exchange of video and music. The video's popularity boom was ahead, although users were already actively sharing viral videos — via bulletin boards or e-mail chains.

On April 23, 2005, a video with the unsympathetic name Me at the Zoo appeared at YouTube's still unknown video host. YouTube allowed a huge number of users to start sharing personal videos, often with meaningless content and without training. Free access and lack of rules brought the hostel the rabid popularity that still exists.

A paradigm shift in the use of video content and the development of the Internet itself made it possible for users to display high-resolution videos. The ability to record quality video on the phone was also a breakthrough. The technology gap led to the emergence of strimming and increased the level of popularity already recognized by everyone at YouTube, as well as the availability of alternative video hosting sites.

Online video became part of everyday life: online meetings at Zoom or Teams with the camera on, training courses on time management, video promotion of a new burger or gym in a nearby area are all new video formats.

What are the types of videos?

Now the video has ceased to be purely entertaining, and there are many criteria for its classification, looking at the basics.

In the first place, the video can be the main activity or an application.

Second, video service models differ by target audience: C2C, B2C, B2B, and even B2D – Business to Developers – services that make it easier for developers to perform online video tasks.

Since the video generates income for its creators around the world, it is divided by the type of monetization as follows: free, conditional, fee-free or subscription model, monetization by advertising model, mixed.

It is clear that the video is also divided into content types: entertainment, and business videos — a type that involves the use of a tool system to build products from the other two categories.

How does today's online video service work?

The online video service is not just a player, but a software platform with complex architecture.

Loading ♪ Processing ♪ Storage ♪ Delivery and Playing ♪

Loading is a set of protocols and tools to support the loading or receiving of a stream signal on a device, including the capture of a video from a camera and the capture of a screen. We used to call it a screen display, but the ways in which the media content is delivered to the service are varied: they include the incoming flows from the surveillance cameras and the incoming flows from the transmissions, and the signal can come from different sources, such as a satellite or as a DVD signal.

The production of the video consists of two phases, conversion, which are optimized in size and other parameters.

Mediasegmentation is the cutting of the video into small pieces of about 1 to 10 seconds, which are transmitted to the player by strimming different protocols. The two most famous are the HLS of Apple and MPEG DASH, which actively uses Google, including for YouTube. Next, the player gathers as a puzzle, video, and at a good level of network operation, we do not see these "shwa".

Player can play in a browser or in an app video file simply by getting a link to it, but modern services often play a playlist of little bits of video. The playlist is called a manifesto. The use of a manifesto is needed to reduce delay and prevent buffering, as well as to allow real-time switching of audio tracks and other functionalities.

If the manifesto contains all the familiar features of the player's view, it allows the quality to be changed to flight depending on the online environment. The viewers often do not think about it, but very often the image at the beginning of the video is not ideal, but after 1 to 2 seconds we get the right quality. This is because the player for the quick start of the video gets the worst quality because it is the easiest. Then, if a stable Internet connection is found, the player starts to download larger and better parts.

Another need for a modern video service is to protect bonus content from unauthorized access, for example, when a browser and player exchange data, using encryption.

The player is also an important component of the video service, a modular product that needs to work on all screens. Accordingly, on mobile devices, the player is written in the language in which the application is written for better performance. The Player now is not just a piece of code, but a separate large software product that includes developed libraries for mobile environments, multiple modules to support different striming protocols, encryption, analysts, playlist support, etc. The Player has its own API for developers and requires regular cross-platform testing. Modern companies that pay attention to the quality of video services even have a dedicated stand that includes 15 to 20 different physical devices — mobile phones, tablets of different versions, generations and vendors, as well as a virtual stand. With each new version of the player, it is tested for compatibility.

The video service also includes storage of downloaded content and view quality formats. The storage must be non-resistant, so several copies of the video are made, and most often a very large file warehouse, distributed between different data centres, is used for several dozen petabytes. For example, we have more than 5 BOPs in a storage facility of this type.

There are two other important components in the video service structure: the CDN and the analyst. The CDN is a network of distributed servers to deliver the video content near the user's path. This is very important because the video content is very heavy.

There is also a separate set of analysts: a consumer analyst and a service quality analyst that evaluates buffering events and download rates, helps improve the service and so on.

It's a common architecture of modern video services, and in reality it's much more complicated because it involves interaction between all these elements through protocols, failure testing, scale and maintenance, support for API, monitoring, regular testing, and so on.

Difficulties in developing a video service

We took a lot of time to study and test the configuration of the system. We started with one architecture, but then when we got to a certain amount, we realized that the architecture had to be completely re-engineered. This is the main challenge of working with video-streaming -- it can be easy and easy at the initial stage, but after a while it turns out that there are serious difficulties in managing a large number of objects. The video work is an unconditional highload. Cataloguing, delivery, trans-coding -- there are all kinds of underwater rocks that you only face when you face real loads. The whole process of re-activating the Kinescope platform took a very long time: two years of re-acting software and five years of hardware.

As a result, we changed the approach to content processing: while we used only CPU coding five years ago, now 90 per cent of the process comes from graphic accelerators, which have made a lot of progress in recent years. By the way, a very large evolution in video services was due to the development of iron: network maps, access to disk space.

In Kinescope, we've been preparing the foundation for our platform for quite a long time because we've been doing our own infrastructure, developing our own network, building service solutions. If we were using some cloud service provider that provides computer rental like Yandex Cloud or AWS, we'd start faster, but business has to realize that in this case there's a loss of control and speed that helps to compete in today's market. The whole business is now trying to cut costs, and even relatively small video costs are putting a heavy burden on the budget of any company, even a large company.

What's the next thing for video services?

Important drivers for the development of video services are now:

  • Connectivity in social media. UGC video has become inextricably linked to social media — Instagram, Facebook, Snapchat, and others. Illustrative content. TED Talks' first performances have appeared in 2006, and since then have become very popular — speakers' performances have gained more than a billion views, and this is only a fraction of the educational market. Entertainment content. Market giants — Netflix, Hulu, ivi — provide comfortable access to more TV shows or films. Flow transmission. Direct transmission now generates an increasing share of traffic, and the ability to conduct them unprofessionally accelerate growth.

In the near future, the video will continue to be linked to the mobile format of content consumption on 4/5G networks, with 70 per cent of the world ' s population having access to mobile communications by 2023.

New interactive tools, not just for the entertainment industry, are constantly emerging. The format of the video is increasingly used for business purposes, so in parallel with the development of the video, online sales in strims, commercial storis, and tape videos are also developing. As early as 2018, Hubspot wrote that 54% of consumers want to see more video content from a brand or company.

The video can tell you the way the text can't. It's always gonna be a bright piece in an endless information stream: a dynamic image, sound, and various effects.