The Accurate Google Cloud Simulator (AGOCS) framework was created to support the testing of complex load balancing strategies without the need for a massive Cloud / cluster environment. AGOCS can simulate the workload of hundreds of thousands of nodes and provides very detailed statistics, such as requested/used CPU Cores, used canonical (kernel) and application memory, used page cache memory, disk I/O times, used local and remote disk space, average cycles per instruction and average memory access per instruction.
KEY FEATURES LIST
- A high-fidelity Cloud workload simulator that has been developed to test schedulers’ prototypes using a realistic workload;
- A modern technology stack which is well-suited to parallel simulations and can create a good foundation for testing prototypes;
- A realistic workload data based on Google’s traces. Data has been additionally analysed and all anomalies cleaned;
- A highly concurrent design, capable of simulating hundreds of thousands of active nodes in a Cloud system;
- The easy configuration of workload events (CSV files) and build-in scenarios (based on Google’s workload traces);
- The easy customisation for all new sources of workload traces;
- A ‘stop and replay’ feature which allows researchers to examine the impact of their strategies on a Cluster state.
WORKLOAD DATA
AGOCS analyses traces from the Google Cluster Data (GCD) project. GCD workload traces are month-long, and contain processing data from a computing cell of ca. 12.5k nodes. Google services are constantly utilised, 24-hours a day, and can be accessed from any location around the globe. As such, they provide a good variety of tasks found within the production environment. Additionally, since GCD traces were cleaned of all anomalies, AGOCS provides only high-quality data.
The AGOCS can provision many of test environments simultaneously.
PERFORMANCE
Given the sheer complexity of workload data, the main requirement for AGOCS was a highly parallel design. AGOCS was built using functional programming concepts with a Scala and Akka Actors / Streams framework. At approximately 300 bytes, this framework has an extremely low overhead per instance. Akka library uses efficient Google’s Protocol Buffers as its default serialisation mechanism for internal communications, i.e. between the cluster nodes. AGOCS was designed for common desktop machines even though it requires loading, and processes a huge amount of workload traces data (191GB of uncompressed data). AGOCS can also multiply built-in scenarios – during testing, it was able to simulate 100k active nodes which is eight times original GCD’s cluster size.
One interesting feature of AGOCS is that it can be paused at any time, allowing users to take a snapshot of current tasks distributions and the state of scheduled jobs. The snapshot files contain all simulation data in a serialised form, meaning that they can be stored and examined later. This approach enables researchers to conveniently and directly compare various scheduling algorithms at any time while they are running.
OTHER SIMULATORS
AGOCS is the most advanced Cloud workload simulator available on the market. In comparison to existing products such as CloudSim and EMUSIM, AGOCS provides more detailed statistics, and can simulate very large instances of hundreds of thousands of nodes.
LICENSING
This unique offering is aimed at researchers and developers of Cloud / cluster systems. If you are interested in licensing AGOCS, please contact us.