In Short
Agora was looking for a partner to migrate their analytics platform from a Hadoop cluster to Google Cloud and to support them in the migration process to also optimise the solution in terms of costs.
Results achieved include: Data from several key sources now available in one place, a unified working environment for analysts, all historical traffic data transferred to the cloud, data management & access procedures created, and unified feed processes for Data Lake and Data Warehouse introduced.
Key benefits reached for the organisation include no infrastructure-related maintenance work anymore, higher computing performance and faster operation, greater stability, on-demand scalability, democratisation of data access & cost optimisation
About the customer
Agora is the 4th largest media group in Poland. Its offerings include the Helios SA cinema chain, the Next Film film production and distribution company, 9 radio stations operating under the Eurozet Group banner, outdoor advertising (AMS SA), respected news media – ‘Gazeta Wyborcza’ and Wyborcza.pl, the Gazeta.pl portal and its services (including Sport.pl, Moto.pl and Plotek.pl), the supra-regional Radio TOK FM, book, music and film publishing, as well as catering operations.
The Challenge
In 2014, Agora began collecting data on service user activity using its own technology.
A Hadoop cluster, data model and data collection mechanisms were developed for data storage and processing.
Hive was chosen for analytical purposes. The business requirement was constant access to up-to-date, aggregated data. Data management processes were built and a number of ETL processes based on it. In addition to traffic data, Hive also stored content metadata from monitored websites and other data from various systems.
The ever-increasing volume of data, the growing reporting needs on the Business side, combining different data sets, maintaining the physical infrastructure and software, addressing errors and failures – these were the main challenges of the on-prem setup.
Concluding, the challenge featured:
- Variable load on the computing unit
- 70 TB of data to be migrated from Hive to BigQuery
- Integration with Kafka and other solutions providing real-time data
- Optimisation of service utilisation
Taking into account the risks and limitations of the current analytical platform, Agora’s Management Board decided to migrate its analytical processes to the Google Cloud. Due to limited experience in the cloud area, Agora was looking for a partner to support employees in the migration process at the stage of modelling the target data structure and creating the architecture of the cloud provisioning system – in order to optimise the above elements of the solution in terms of costs.
Devoteam G Cloud was a reliable partner in areas such as training, knowledge transfer and DevOps for our team responsible for migrating a large-scale data warehouse from an on-prem solution to the cloud. It was a pleasure to work with Devoteam engineers.
The Solution
The Devoteam team prepared a concept using Managed and Serverless solutions that would guarantee that Agora S.A.’s analytics teams would retain their existing style of working.
Following client approval, a project was implemented to migrate the analytics platform to the Google cloud, using components such as BigQuery, Google Cloud Storage, Cloud Composer (Airflow), Cloud Functions or Dataflow.
The cooperation between Devoteam and the Agora S.A. team in the project has allowed for a smooth transfer of knowledge, enabling Agora S.A. employees to successfully carry out further development work, as well as to maintain the Big Data environment created together with us. The partnership approach and very good preparation of the Agora S.A. team allowed all the necessary work to be carried out as planned.
The Result
The results we achieved:
- All historical traffic data was transferred to the cloud
- A framework / system of procedures related to data management and access to data was created
- Unified feed processes for Data Lake and Data Warehouse were introduced
- Data from several key sources is now available in one place
- The working environment for analysts has been unified
Key benefits for the organisation:
- No infrastructure-related maintenance work
- Higher computing performance / faster operation
- Greater stability
- On-demand scalability
- Democratisation of data access
- Cost optimisation
Currently, the Agora S.A. team is supported by Devoteam in supporting the continuity of Google Cloud services.
Need help from an expert Google Cloud partner?
