SchIaaS Documentation

SchIaaS allows to simulate IaaS with SimGrid.

While SimGrid implements hypervisor level functionalities (e.g. start and stop VMs), SchIaaS implements cloud level functionalities (e.g. run and terminate instances).

It supports main VM management functions:

• run, terminate, suspend, resume and describe instances;
• description of available resources;
• image and instance types management;
• VM placement on the clusters;
• boot and other VM life-cycle processes;
• cloud storage management.

For now, it does not support other cloud functionalities, such as network management.

SchIaaS is a place-holder interface: every operation is actually handle by an engine, and new engines can be developped and plugged in. This allows SchIaaS applications to be compatible with any cloud internals, or even use different internals together in a multi-cloud context.

For now, two engines are available and fully functional:

• RICE: Reduced Implementation of Compute Engine
• RISE: Reduced Implementation of Storage Engine

Running SchIaaS

Prerequisite

SchIaaS runs upon SimGrid. Make sure to enable Java support:

git clone git://scm.gforge.inria.fr/simgrid/simgrid.git
cd simgrid
cmake -Denable_java=on .
make
sudo make install

You can also get the Simgrid jar from the simgrid website.

Retrieving and Installing SchIaaS

The last version of SchIaaS can be retrieved from the git repository :

git clone https://gforge.inria.fr/git/schiaas/schiaas.git
or
git clone git://scm.gforge.inria.fr/schiaas/schiaas.git

Please note the code is still unstable.

Afterward, you can build SchIaaS:

cd schiaas
cmake .
sudo make

If cmake complains about the location simgrid.jar, make sure to set SIMGRID_JAR_PATH:

export SIMGRID_JAR_PATH=/path/to/simgrid.jar/

For instance:

export SIMGRID_JAR_PATH=/usr/local/java/

If make complains about ggplot2, make sure to install it:

sudo apt-get install r-cran-ggplot2
@endconde

The build produces a schiaas.jar file in the bin/ directory under the current working directory. 

This schiaas.jar is to be added to the list of the java and jar files of your own programs when compiling, as shown in the \ref tutorial_running. 



You may also get a pre-compiled schiaas.jar <a href="./schiaas.jar">here</a>.




@section tutorial Tutorial: the master-slave example

@subsection tutorial_overview Overview of the Example


For a better understanding of this tutorial we recommend that you read the SimGrid's Java MSG master-slave example. 
The following is essentially a summary of the differences between the SimGrid's way of describing a master-slave computation
and the SchIaaS's one.
The main difference between this cloud scenario and the original SimGrid example lies in the 
fact that slave instances are deployed on the fly, while in the original SimGrid example, 
slaves to launch were statically declared in the deployment configuration.


Recall that setting up this example requires :
- configuration (XML) files to describe the environment and the mapping of processes to hosts  
- Java programs to describe the behavior of the processes: 
        - **Masterslave.java** is the orchestrator: it reads the configuration files and runs the simulation
        - **Master.java** describes the Master process's behavior: 
   it creates a number of slave processes specified in the deployment configuration file, and do clever things with slave processes.
   In the SimGrid Masterslave example, the master is just sending data to slaves, while we demonstrate additional cloud-related features like 
   migration of VM in this SchIaaS example.
        - **Slave.java**: the Slave behavior (unchanged from simGrid example) which simply receives data from the Master.

These files can be found in the distribution under *examples/cloudmasterslave/*



@subsection tutorial_setup Simulation setup

Recall that SimGrid uses a *platform file* to describe the set of all hardware devices available,
and a *deployment file* to tell onto which of these devices the different processes run. 
 
For a schiaas simulation, the platform file is the one you would normally use for a SimGrid simulation. 

\verbinclude "cloudmasterslave/companionFiles/platform.xml"

The deployment file is also similar excepted that it does not need to include the placement of slave processes anymore, as slaves processes will be bound to instances at runtime.

\verbinclude "cloudmasterslave/companionFiles/deploy.xml"


Last, a new configuration file is introduced to declare the cloud configuration, cloud.xml:

\verbinclude "cloudmasterslave/companionFiles/cloud.xml"

Cloud files are validated using <a href="./cloud.xsd">cloud.xsd</a> found in the SchIaaS sources.

@subsubsection cloud_tag Cloud tag

The cloud tag must have the following attributes:
- id: the id of the cloud or of the region;

You can declare as many clouds as needed, and clouds can also be used to model regions.

Inside the cloud element, you must define storage and compute.


@subsubsection storage_tag Storage tag

The storage tag must have the following attributes:
- id: the id of the storage;
- engine: the engine of the storage;

It must also include one config tag, giving any information needed by this engine. For instance, RICE only needs one controller, which will store and distribute the data.

As many storages as needed can be declared, and they must be declared before the compute tag. 


@subsubsection compute_tag Compute tag

The compute tag must have the following attributes:
- engine: the engine of the compute;

It must also include the following tags:

- config: any information needed by this engine. For instance, RISE needs: 
  - one controller, which will distribute the commands; 
  - an image_storage; an image_caching strategy, that can be ON, OFF, or PRE; 
  - an inter_boot_delay, that is the delay between two consecutive VM boots on one given host.

- instance_type, having some attributes describing the type of instance;

- image, having two attributes:
  - id;
  - size;

- host or cluster, to declare the hosts usable to run instance.


@subsection tutorial_code Java Code

@subsubsection tutorial_initialization Initialization

As previously explained, we need an orchestrator to read and setup the environment. 
This is the *Masterslave.java* file in this example. 
In addition, it has to initialize SchIaaS though SchIaaS.init(), its argument being the cloud configuration file. 
@code
import org.simgrid.schiaas.SchIaaS;
[...]

        /* construct the platform and deploy the application */
        Msg.createEnvironment(args[0]);
        Msg.deployApplication(args[1]);
        
        /* construct the cloud and deploy the associated processes */
        SchIaaS.init(args[2]);


        /*  execute the simulation. */
        Msg.run();

Instances management

Master.java* controls the application launch. Given that we simulate virtualization through VM images, Master.java contains a couple of changes as compared to the original Master.java of SimGrid. It must declare an instance image and run one instance per slave:

import org.simgrid.schiaas.*;
[...]
        Msg.info("Hello! Got "+  slavesCount + " slaves and "+tasksCount+" tasks to process");
        // Retrieve the Compute module of MyCloud
        Compute myCompute = SchIaaS.getCloud("myCloud").getCompute();
        
        // Run one instance per slave on myCloud
        String[] slaveInstancesId = myCompute.runInstances("myImage", "small", slavesCount);

Afterwards, slaveInstancesId contains instances' id, which are the name of SimGrid's VMs and can be used just like any SimGrid host.

Please note that if you ask for more instances than your cloud can provide, runInstances and runInstance will return null. Careful users will check how many instances can be ran:

for (InstanceType instanceType : myCompute.describeInstanceTypes()) {
                        Msg.info(instanceType.getId()+ ": "
                                        + myCompute.describeAvailability(instanceType.getId()));
}

Before using theses instances, you must wait for the end of their boot process, otherwise, the bound processes will start immediately and concurrently to the boot process, mistaking both your process running date and the boot duration. Then, you can bind the slave process to the running instance:

for (int i=0; i<slavesCount; i++) {
        Msg.info("waiting for boot");
        while (myCompute.describeInstance(slaveInstancesId[i]).isRunning()) {
                waitFor(10);
        }
        
        Msg.info("Starting a slave on "+myCompute.describeInstance(slaveInstancesId[i]).getName());
        String [] slaveArgs = {""+i};
        Slave s = new Slave(myCompute.describeInstance(slaveInstancesId[i]),
                                                "slave_"+i,slaveArgs);
        s.start();
}

During the execution, you can suspend and resume instances:

waitFor(150);
Msg.info("Suspending " + slaveInstancesId[0]);
myCompute.suspendInstance(slaveInstancesId[0]);
waitFor(200);
Msg.info("Resuming " + slaveInstancesId[0]);
myCompute.resumeInstance(slaveInstancesId[0]);

Finally, you can terminate instances one by one:

myCompute.terminateInstance(oneInstanceId);

Then, you have to terminate SchIaaS to clean-up the simulation:

SchIaaS.terminate();

Please note that this will also terminate all the instances that are still running.

Data management

You can also manipulate some data:

Storage myStorage = SchIaaS.getCloud("myCloud").getStorage("myStorage");

Data someData = new Data("someData", 1e9);

Msg.info("Store some data");
myStorage.put(someData);

Msg.info("Check whether the data transfer is complete: " +
                myStorage.isTransferComplete("someData"));

Msg.info("Retrieve some stored data");
someData = myStorage.get("someData");

Msg.info("Retrieve some data that has not been stored");
Data unstoredData = new Data("unstoredData", 2e9);
someData = myStorage.get(unstoredData);

Msg.info("List the stored data");
Map <String, Data> storedData = myStorage.list();       //1st way
myStorage.ls();                                                                         //2d way

Msg.info("Delete some data");
myStorage.delete("someData");

Reports

And last, you can obtain cloud usage report by modifying Masterslave.java:

/* execute the simulation */
Msg.run();        

/* print cloud reports */
Msg.info("Cloud details\n");
for (Cloud cloud : SchIaaS.getClouds())  {
        Msg.info("Cloud:"+cloud.getId());
        for (Instance instance : cloud.getCompute().describeInstances()) {
                Msg.info(" - "+instance);
        }
}

Compiling and Running the Example

Compiling the cloudmasterslave example

• First, make sure you can access simgrid.jar. We advise to set $SIMGRID_JAR_PATH to the directory where simgrid.jar has been installed. We assume here

  $ export SIMGRID_JAR_PATH=/usr/local/simgrid/java

• Second, let us compile the source files and direct the resulting class files to the bin/ directory. The source Java files are located in schiaas/examples/cloudmasterslave Let us make schiaas/examples our current working directory.

  $ cd examples
  $ javac -d ../bin/ -cp ${SIMGRID_JAR_PATH}/simgrid.jar:../bin/schiaas.jar cloudmasterslave/*.java

• Then make a jar of the class files:

  $ cd ../bin
  $ jar cvf cloudmasterslave.jar cloudmasterslave/*.class 

Running the cloudmasterslave example

• You are now ready to run the example. For convenience, you can copy the configuration files to the current directory.

  $ cp ../examples/cloudmasterslave/companionFiles/{platform,deploy,cloud}.xml .
  $ java -cp ${SIMGRID_JAR_PATH}/simgrid.jar:schiaas.jar:cloudmasterslave.jar cloudmasterslave.Masterslave platform.xml deploy.xml cloud.xml