Big data analytics for gaming company

Our client runs a popular mobile gaming company that launched an iOS and Android gaming application allowing multiplayer interaction. However, with the rapid increase in number of users and data volume, their existing analytics platform faltered. It could not handle the data load and provide accurate insights via visualization and reporting. They needed a new, scalable analytics platform capable of:

• Visualizing and reporting the most current data
• Tracking KPIs to understand their users better and optimize monetization
• Monitoring of the platform’s health state.

To address the client’s challenges, we proposed a comprehensive solution that involved migrating from Postgres to Snowflake, connecting data sources, and building a platform that enables advanced analytics, reporting, and visualization to monitor gamer activity, new users, user retention, and in-game purchases. We also proposed a data orchestration process using Airflow and a data transformation process to optimize the data storage by converting raw CSV data from sources to optimized Parquet format.

Here is what we did:

Implementation

We led the end-to-end implementation, including data engineering, platform migration, model development, and integration. Our solution ensures system stability and adaptability. It can handle adding or modifying data sources or changing data structures automatically.

Database migration and continuous data stream

To achieve improved querying speed at scale, we successfully migrated terabytes of data from Postgres to Snowflake’s optimized schemas. The migration process was orchestrated using Airflow, ensuring consistency and reliability throughout the transition.

To enhance data organization and storage, we leveraged an AWS data lake, allowing for efficient management and retrieval of information. Snowflake’s scalable storage solution also enabled us to define external tables over our S3 Parquet files, eliminating the need to move or duplicate data. This architecture facilitated seamless querying of petabytes of operational data and enabled analysis through SQL as well as visualization using connected BI tools.

In addition, Snowflake’s performance, manageability, and pay-per-use billing model create a robust and cost-effective solution for handling large volumes of data.

Data orchestration

We have developed an incremental data flow that merges source CSV files and converts them to an optimized Parquet format, partitioned by date, before populating them into Snowflake. This process is flexible and can adapt automatically to any changes in the source structure or adjustments.

Analytical platform development

We built a robust analytical platform that empowered the client to perform advanced analytics, generate comprehensive reports, and visualize gamer activities in near real-time. The platform enables the client to monitor user behavior, track repeat usage, analyze the lifespan of active users, and gain insights into in-app purchases, among other metrics.

Optimization of monetization strategy

We analyzed the pricing of coin bundles via machine learning models. We used historical purchase and catalog data to predict the impact of small bundle price changes on expected daily revenue. We linked predicted changes to allowed percentage price changes for bundles based on frequent prediction ranges. This enabled us to maximize revenue while managing acceptable risk.

Our optimizations delivered actionable insights at scale to drive critical decisions. The implemented enhancements ensure our client’s competitive edge through ever-increasing data-driven intelligence. They can now effortlessly handle the increased data volume, gaining valuable insights in real-time.