DPCS@IN3-UOC

  • Increase font size
  • Default font size
  • Decrease font size

LSim tutorial

E-mail Print PDF

LSim tutorial

In this tutorial we will learn how to adapt a simple distributed application to use LSim and also  a few basic characteristics that LSim offers. A basic knowledge of the LSim is assumed. A description of the LSim can be found here.

In chapter 1 the initial application is described and chapters 2 to 5 describe the process to adapt the initial application so it can be executed in a distributed environment using the LSim.

Finally chapter 6 explains how to run the application in a local environment simulating a distributed environment. This may be useful to test applications before deploy them in a distributed environment.

You can download the code used on the tutorial here.

1. Initial application

Our application is composed by two types of components:

  • client: sends a number. There may be several instances of this type.
  • server: receives the numbers from clients and sums them up. There's only one instance of this type. It waits to receive as many numbers as clients.

Client code

package application;
import java.io.DataOutputStream;
import java.io.IOException;
import java.net.Socket;
import java.net.UnknownHostException;
public class Application implements ApplicationManager {
    public static void main(String argv[]) {
        System.out.println("Application running");
        
        String str = argv[0];
        System.out.println("number to send: " + str);
        Socket clientSocket;
        try {
            clientSocket = new Socket("localhost", 6789);
            DataOutputStream outToServer = new DataOutputStream(
                    clientSocket.getOutputStream());
            outToServer.write(new Integer(str).intValue());
            clientSocket.close();
        } catch (UnknownHostException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Server code

package server;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.ServerSocket;
import java.net.Socket;
public class Server implements ApplicationManager {
 
    public static void main(String argv[]) {
        int total = 0;
        int n = Integer.parseInt(argv[0]);
        ServerSocket welcomeSocket;
        try {
            welcomeSocket = new ServerSocket(6789);
            for (int i = 0; i < n; i++) {
                Socket connectionSocket = welcomeSocket.accept();
                BufferedReader inFromClient = new BufferedReader(
                        new InputStreamReader(connectionSocket.getInputStream()));
                total += inFromClient.read();
            }
            System.out.println("Total: " + total);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}
 

2. Adapting the application to run with LSim

LSim library is packed in the file lsim-library.jar. This library must be added to the project when developing or in the classpath on compiling. The packet lsim-commons.jar is also needed, it includes several classes used to create the running environment.

Client

First steps

1. The class Application needs to implement ApplicationManager class that is located on lsim.application.ApplicationManager.

2. Copy the code on main() method to method start(LSimDispatcherHandler dispatcher). This way the application can be executed with LSim and using main method also.

When this two steps are done, application code should be like this (added/modified code is in blue font color):

package application;
import java.io.DataOutputStream;
import java.io.IOException;
import java.net.Socket;
import java.net.UnknownHostException;
import java.util.List;
import lsim.LSimDispatcherHandler;
import lsim.application.ApplicationManager;
import lsim.application.handler.DummyHandler;
import lsim.application.handler.InitHandler;
import lsim.worker.LSimWorker;

public class Application implements ApplicationManager { public static void main(String argv[]) { System.out.println("Application running"); String str = argv[0]; System.out.println("number to send: " + str); Socket clientSocket; try { clientSocket = new Socket("localhost", 6789); DataOutputStream outToServer = new DataOutputStream( clientSocket.getOutputStream()); outToServer.write(new Integer(str).intValue()); clientSocket.close(); } catch (UnknownHostException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } }
    @Override
    public boolean isAlive() {
        // TODO Auto-generated method stub
        return true;
    }

    @Override
    public void start() {
       // TODO Auto-generated method stub
    }

    @Override
    public void start(LSimDispatcherHandler dispatcher) {
       System.out.println("Application running");
       String str = argv[0];
       System.out.println("number to send: " + str);
       Socket clientSocket;
       try {
           clientSocket = new Socket("localhost", 6789);
           DataOutputStream outToServer = new DataOutputStream( clientSocket.getOutputStream());
           outToServer.write(new Integer(str).intValue());
           clientSocket.close();
       } catch (UnknownHostException e) {
           e.printStackTrace();
       } catch (IOException e) {
           e.printStackTrace();
       }
    }

    @Override
    public void stop() {
       // TODO Auto-generated method stub
    }
}


3. The following variables should be declared in method start in order to add a Worker role to the application.

LSimWorker lsim = new LSimWorker();
lsim.setDispatcher(dispatcher);

Init

4. Application should wait for init information and parameters from coordinator. Each application instance will receive the parameters sent by the coordinator through a handler.

InitHandler init = new InitHandler(lsim, 5);
lsim.init(init);

In this code an InitHandler object included on LSim library is used. It has two parameters: (a) Lsim instance and (b) the maximum time in minutes that the experiment should last before ending. Time should be optimistically estimated because it includes the initialization and starting time of all instances.

5. The method getParameters() allows to obtain the received parameters needed to initialize the application instance. First (and in the same order) there will be the parameters on specifications (XML) of the experiment and send to the Launcher. Then the parameters set by the coordinator will follow.

List <Object> param = init.getParameters(); 
String str = param.get(0).toString();

In this example, there is only one parameter, the number that the application has to send to the server.

start

6. Once initialized, the applications gets blocked until it receives the command to start execution.

lsim.start(new DummyHandler());

The application doesn't need any parameter to start, so here the DummyHandler is used. This handler is also part of the library and its only purpose is to wait for the coordinator to signal the start.

7. The application should finish invoking stop() method. This command is needed for coordination purposes and not in the scope of this tutorial.

lsim.stop(new DummyHandler());

After this steps are done, code on method start should look like this (added/modified code is in blue font color):

    @Override
    public void start(LSimDispatcherHandler dispatcher) {
        System.out.println("Application running");
        
        LSimWorker lsim = new LSimWorker();
        lsim.setDispatcher(dispatcher);
        // init InitHandler init = new InitHandler(lsim, 5);
        lsim.init(init);
        // getting parameters List<Object> param = init.getParameters();
        String str = param.get(0).toString();
System.out.println("number to send: " + str); Socket clientSocket; try { clientSocket = new Socket("localhost", 6789); DataOutputStream outToServer = new DataOutputStream( clientSocket.getOutputStream()); outToServer.write(new Integer(str).intValue()); clientSocket.close(); } catch (UnknownHostException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); }         lsim.stop(new DummyHandler()); }

Server

8. Just as we've seen, the server adaptation will result in this:

package server;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.List;
import lsim.LSimDispatcherHandler;
import lsim.application.ApplicationManager;
import lsim.application.handler.DummyHandler;
import lsim.application.handler.InitHandler;
import lsim.application.handler.ResultHandler;
import lsim.worker.LSimWorker;
public class Server implements ApplicationManager { public static void main(String argv[]) { int total = 0; int n = Integer.parseInt(argv[0]); ServerSocket welcomeSocket; try { welcomeSocket = new ServerSocket(6789); for (int i = 0; i < n; i++) { Socket connectionSocket = welcomeSocket.accept(); BufferedReader inFromClient = new BufferedReader( new InputStreamReader(connectionSocket.getInputStream())); total += inFromClient.read(); } System.out.println("Total: " + total); } catch (IOException e) { e.printStackTrace(); } }

    @Override public boolean isAlive() {
         // TODO Auto-generated method stub
         return true;
    }

    @Override
public void start() {
        // TODO Auto-generated method stub
    }

    @Override public void start(LSimDispatcherHandler dispatcher) {
        LSimWorker lsim = new LSimWorker();
        lsim.setDispatcher(dispatcher);
        int total = 0;
        // init InitHandler init = new InitHandler(lsim, 5);
        lsim.init(init);

        // getting parametres
        List<Object> param = init.getParameters();
        int n = Integer.parseInt(param.get(0).toString());
        System.out.println("I've recieved: " + n);

        // Open socket before starting
        ServerSocket welcomeSocket = null;
        try {
            welcomeSocket = new ServerSocket(6789);
        } catch (IOException e1) {
            e1.printStackTrace();
        }

        // start
        lsim.start(new DummyHandler());
        try {
            for (int i = 0; i < n; i++) {
                Socket connectionSocket = welcomeSocket.accept();
                BufferedReader inFromClient = new BufferedReader( new InputStreamReader(connectionSocket.getInputStream()));
                int valueFromApp = Integer.parseInt(inFromClient.readLine().trim());
                total += valueFromApp;
        }
        System.out.println("Total: " + total);
        welcomeSocket.close();
        } catch (IOException e) {

            e.printStackTrace();
        }
    }

    @Override public void stop() {
        // TODO Auto-generated method stub
    }
}

8. We want that the server sends multiple results to be evaluated. First it will send partial results to an evaluator, and then it will send the sum to another evaluator.

lsim.sendResult("checkServer", new ResultHandler(valueFromApp));

Here we use ResultHandler avaliable in LSim library. The handler has one parameter where we have to include the object result that will be passed to evaluator.

9. We want to send the sum of all received numbers to another evaluator. Calling sendResults will use the default evaluator named.

// send end result
lsim.sendResults(new ResultHandler(total));

10. Finally, the resultant code of method start will be like this:

    @Override
    public void start(LSimDispatcherHandler dispatcher) {
        LSimWorker lsim = new LSimWorker();
        lsim.setDispatcher(dispatcher);
        int total = 0;
        // init
        InitHandler init = new InitHandler(lsim, 5);
        lsim.init(init);
        // getting parametres
        List<Object> param = init.getParameters();
        int n = Integer.parseInt(param.get(0).toString());
        System.out.println("I've recieved: " + n);
        
        // Open socket before starting
        ServerSocket welcomeSocket = null;
        try {
            welcomeSocket = new ServerSocket(6789);
        } catch (IOException e1) {            
            e1.printStackTrace();
        }        
        
        // start
        lsim.start(new DummyHandler());
        
        try {
            for (int i = 0; i < n; i++) {
                Socket connectionSocket = welcomeSocket.accept();
                BufferedReader inFromClient = new BufferedReader(
                        new InputStreamReader(connectionSocket.getInputStream()));
                int valueFromApp = Integer.parseInt(inFromClient.readLine().trim());
                total += valueFromApp;
                // send partial result
                lsim.sendResult("checkServer", new ResultHandler(total));
            }
            System.out.println("Total: " + total);
            
            welcomeSocket.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
        // send end result
        lsim.sendResults(new ResultHandler(total));
        lsim.stop(new DummyHandler());
    }

3. Adding a coordinator

We will implement a simple coordinator for our experiment. The coordinator is responsible for receive the experiment specification and send to workers all parameters that they need. This is handled by the library so we only need to call init, start and stop methods.
Here is our coordinator:

package coordinator;
import lsim.LSimDispatcherHandler;
import lsim.application.ApplicationManager;
import lsim.application.handler.DummyHandler;
import lsim.application.handler.InitHandler;
import lsim.coordinator.LSimCoordinator;
public class TutorialCoordinator implements ApplicationManager{
    @Override
    public boolean isAlive() {
        // TODO Auto-generated method stub
        return true;
    }
    @Override
    public void start() {
        // TODO Auto-generated method stub
        
    }
    @Override
    public void start(LSimDispatcherHandler disp) {
        
        System.out.println("Set dispatcher");
        LSimCoordinator lsim=new LSimCoordinator();
        lsim.setDispatcher(disp);
    
        System.out.println("Before init");
        InitHandler init=new InitHandler(lsim,30);
        lsim.init(init);
        System.out.println("Before start");
        
        lsim.start(new DummyHandler());
        
        lsim.stop(new DummyHandler());
        
        
        
    }
    @Override
    public void stop() {
        // TODO Auto-generated method stub
        
    }
}

4. Adding a partial evaluator

This evaluator will receive the numbers received by the server from clients. It will evaluate that the received numbers sum up the same that expected.

package evaluator;
import lsim.LSimDispatcherHandler;
import lsim.application.ApplicationManager;
import lsim.application.handler.DummyHandler;
import lsim.application.handler.InitHandler;
import lsim.evaluator.DefaultEvaluatorResultHandler;
import lsim.evaluator.LSimEvaluator;
public class PartialEvaluator implements ApplicationManager {
    @Override
    public boolean isAlive() {
        // TODO Auto-generated method stub
        return true;
    }
    @Override
    public void start() {
        // TODO Auto-generated method stub
    }
    @Override
    public void start(LSimDispatcherHandler disp) {
        process(disp);
    }
    public void process(LSimDispatcherHandler hand) {
        LSimEvaluator lsim = new LSimEvaluator();
        lsim.setDispatcher(hand);
        InitHandler init = new InitHandler(lsim, 5);
        lsim.init(init);
        
        //Set result handler rh to default (receive results and return one result a time)
        DefaultEvaluatorResultHandler rh = new DefaultEvaluatorResultHandler();
        //Must be done before start
        lsim.setResultHandler(rh);
        lsim.start(new DummyHandler());
        System.out.println("New partial handler");
        PartialEvaluatorHandler handler = new PartialEvaluatorHandler();
        System.out.println("Call sendResult with partialEvaluatorHandler");
        int obtainedResults = 0;
        int total = 0;
        while (obtainedResults < 2) {
            lsim.getResult(handler);
            Integer value = handler.getValue();
            int intValue;
            if (value != null) {
                intValue = handler.getValue();
                // do somethig with value
                obtainedResults ++;
                System.out.println("back from sendResult, get value: " + value);
                total += value;
            }
                        
        }
        if (total == 12){
            System.out.println("Get correct values!");
        } else {
            System.out.println("Error! Values are not correct!");
        }
        
        lsim.Finish();
    }
    @Override
    public void stop() {
        // TODO Auto-generated method stub
    }
}

There's more advanced things on this evaluator like the result handler DefaultEvaluatorResultHandler but they are out of the scope of this tutorial. The main idea is that it is posible to implement the way that the results are received. In order to receive a result  each time that a worker sends it, we can use the default result handler avaliable on LSim library.

The other handler, PartialEvaluatorHandler, just stores and returns the received numbers:

package evaluator;
import lsim.application.handler.Handler;
public class PartialEvaluatorHandler implements Handler {
    
    private Integer value;
    @Override
    public Object execute(Object obj) {
        
        value = (Integer) obj;        
        return value;
    }
    public Integer getValue() {
        return value;
    }
}

5. Adding the end evaluator

We want to evaluate that the sum done by the server is correct and this will be the work of the end evaluator.

package evaluator;
import lsim.LSimDispatcherHandler;
import lsim.application.ApplicationManager;
import lsim.application.handler.DummyHandler;
import lsim.application.handler.InitHandler;
import lsim.evaluator.DefaultEvaluatorResultHandler;
import lsim.evaluator.LSimEvaluator;
public class EndEvaluator implements ApplicationManager {
    @Override
    public boolean isAlive() {
        // TODO Auto-generated method stub
        return true;
    }
    @Override
    public void start() {
        // TODO Auto-generated method stub
    }
    @Override
    public void start(LSimDispatcherHandler disp) {
        process(disp);
    }
    public void process(LSimDispatcherHandler hand) {
        LSimEvaluator lsim = new LSimEvaluator();
        lsim.setDispatcher(hand);
        InitHandler init = new InitHandler(lsim, 5);
        lsim.init(init);
        // Set result handler to standar (receive results and return one result
        // a time)
        DefaultEvaluatorResultHandler rh = new DefaultEvaluatorResultHandler();
        // Must be done before start
        lsim.setResultHandler(rh);
        lsim.start(new DummyHandler());
        System.out.println("Creem el handler");
        EndEvaluatorHandler handler = new EndEvaluatorHandler();
        
        lsim.getResult(handler);
        Integer res = handler.getValue();
        if (res == 12){
            System.out.println("Correct result received!!");
        } else {
            System.out.println("Incorrect result received!!");
        }
        lsim.Finish();
        
    }
    @Override
    public void stop() {
        // TODO Auto-generated method stub
    }
}

The code for the EndEvaluatorHandler is the same as the PartialEvaluatorHandler:

package evaluator;
import lsim.application.handler.Handler;
public class EndEvaluatorHandler implements Handler {
    
    private Integer value;
    @Override
    public Object execute(Object obj) {
        value = (Integer) obj;        
        return value;
    }
    public Integer getValue() {
        return value;
    }
}

6. Create the specification

The specification contains the description of the multiple parts of the experiment: coordinator, workers, evaluators and exception managers. This is the specification that you can find in the launcher folder of the tutorial file, named tutorialLSim.xml:

<?xml version="1.0" encoding="UTF-8" ?>
<Specification>
    <experimentId>tutorialLSim</experimentId>
    <Coordinators>
        <num>1</num>
        <codeRef>lsim.LSimDispatcherHandler::/home/manel/lsim/tutorial/elements/coordinator::coordinator.sh
        </codeRef>
        <timeInit>5</timeInit>
        <timeEval>1</timeEval>
        <param>2</param>
    </Coordinators>
    <ExceptionManager>
        <class>lsim.exceptions.remote.RemoteExceptionManager</class>
        <properties>
            <property name="host" value="localhost" />
            <property name="port" value="9989" />
        </properties>
    </ExceptionManager>
    <Evaluators>
        <Evaluator>
            <codeRef>lsim.LSimDispatcherHandler::/home/manel/lsim/tutorial/elements/partial_evaluator::partial_evaluator.sh
            </codeRef>
            <idEvaluator>checkServer</idEvaluator>
            <idTest>1</idTest>
            <Node>
                <idNode>server</idNode>
                <nMess>1</nMess>
            </Node>
        </Evaluator>
        <Evaluator>
            <codeRef>lsim.LSimDispatcherHandler::/home/manel/lsim/tutorial/elements/evaluator::evaluator.sh
            </codeRef>
            <idTest>1</idTest>
            <Node>
                <idNode>server</idNode>
                <nMess>1</nMess>
            </Node>
        </Evaluator>
    </Evaluators>
    <Workers>
        <Worker>
            <num>1</num>
            <idWorker>server</idWorker>
            <codeRef>lsim.LSimDispatcherHandler::/home/manel/lsim/tutorial/elements/server::server.sh
            </codeRef>
            <param>2</param>
        </Worker>
        <Worker>
            <num>1</num>
            <idWorker>application1</idWorker>
            <codeRef>lsim.LSimDispatcherHandler::/home/manel/lsim/tutorial/elements/application::application.sh
            </codeRef>
            <param>5</param>
        </Worker>
        <Worker>
            <num>1</num>
            <idWorker>application2</idWorker>
            <codeRef>lsim.LSimDispatcherHandler::/home/manel/lsim/tutorial/elements/application::application.sh
            </codeRef>
            <param>7</param>
        </Worker>
    </Workers>
</Specification>

We defined a coordinator, two evaluators, a worker that will be the server and two workers that will be client applications. 

codeRef must be defined for each element. It indicates what have to be executed for the element that we are defining.
The codeRef of the coordinator is:

<codeRef>lsim.LSimDispatcherHandler::/home/manel/lsim/tutorial/elements/coordinator::coordinator.sh</codeRef>

It's always the same structure:

lsim.LSimDispatcherHandler::directory_path::script_name.sh

Where the first part is always the same, directory path is where the element will be found and script_name.sh is the script that will be executed.
The defined path is according the remote environment where the application will be executed. We have defined an directory structure that will be explained in the next section.
The content of the script coordinator.sh is:

java -cp libraries/*:tutorialLsim.jar dispatcher.application.ApplicationMain coordinator.txt

The same for the other elements, we just change the name for the log coordinator.txt to the desired one.

7. Running the experiment in a local environment

In order to test the application in a local environment we define the following structure:

- tutorial
    - applications //Here the execution folders will be created
    - dispatcher
        - dispatcher.jar
        - dispatcher.sh / dispatcher.bat
            // Execution script for the dispatcher
            java -cp lib/*:dispatcher.jar dispatcher.test.runenvironment.DispatcherRunTest log.txt
        - configDispRun.properties
            // Dispatcher configuration
            numApp=10
            port=9001
            Storage=dispatcher.storage.StorageRunEnvironment
            folder=/path_to_LSim/tutorial/applications
            appPorts=9002-9010
            // folder must point to applications folder
        - configAPI.properties
            rmiport=9010
            host=localhost
            name=CoDeS
            lhost=sd2.uoc.edu
        - lib
            {needed libraries*}
    - launcher 
        - tutorialLsim.xml // Experiment specification that we've seen
        - localLauncher.sh / localLauncher.bat 
            // Execution script for the local launcher indicating where's specification file
            java -cp lib/*:launcher.jar launcher.test.LauncherTest tutorialLsim.xml 
        - configAPI.properties
            rmiport=9010
            host=localhost
            name=CoDeS
            lhost=sd2.uoc.edu
        - launcher.jar
        - lib
            {needed libraries*}
    - CoDeS
        - LocalCodes.jar
        - codesAPILocal.sh / codesAPILocal.bat
            java -cp lib/*:codesAPI.jar localCodesPack.LocalCodesTest > log.txt 
        - config.properties
        - configAPI.properties
        - lib
            FreePastry-2.1.jar
            codes.jar
    - elements
        - coordinator
            - tutorialLsim.jar
            - application.properties
            - configAPI.properties
            - coordinator.sh / coordinator.bat
            - libraries
                {needed libraries*}
        - partialEvaluator
            - tutorialLsim.jar
            - application.properties
            - evaluator.sh / evaluator.bat
            - libraries
                {needed libraries*}
        - endEvaluator
            - tutorialLsim.jar
            - application.properties
            - evaluator.sh / evaluator.bat
            - libraries
                {needed libraries*}
        - application
            - tutorialLsim.jar
            - application.properties
            - application.sh / application.bat
            - libraries
                {needed libraries*}
        - server
            - tutorialLsim.jar
            - application.properties
            - server.sh / server.bat
            - libraries
                {needed libraries*}

* Needed libraries must be copied where indicated:

dispatcher.jar
FreePastry-2.1.jar
codesAPI.jar
jdom.jar
launcher.jar
lsim-library.jar
lsim-commons.jar
lsim-exceptions.jar
runenvironment.jar
codesAPI.jar
connector-mina.jar
mina-core.jar
commons-io.jar
slf4j-simple.jar
slf4j-api.jar

When we have created all this structure, the scripts and libraries are on their location we can try to launch the experiment. This are the steps to run the experiment:

  1. Execute resource pool:
    in codes foler, run script codesAPILocal.sh/.bat
  2. Run the dispatcher:
    in dispatcher folder, run script dispatcher.sh/.bat. We can see the ouput of the dispatcher with the command tail -f log.txt
  3. Launch the experiment with launcher:
    in launcher folder, run the script localLauncher.sh/.bat. We will see the progress of the execution.

If the execution is successful we can see the results of the execution on each folder in the applications folder.

This folders have the name TutorialLsim[timestamp]X0nom. There should be a folder for each element: server, application1, application2, coordination, checkServer and endEvaluator.

In evaluators folders we can see the results of the evaluation.

A zip file containing the folder structure and needed libraries and files defined in this section can be found here (windows version) or here (linux version). After download and unzip the files, the path to the LSim tutorial folder must be changed in the following files:

  • tutorialLsim.xml (in folder launcher)
  • configDispRun.properties (in folder dispatcher)

And the files in the folder "libreries" must be copied to the following folders:

  • dispatcher/lib
  • launcher/lib
  • elements/coordinator/libraries
  • elements/partialEvaluator/libraries
  • elements/endEvaluator/libraries
  • elements/server/libraries

Then the folder libraries and its content can be deleted.