Changes between Version 16 and Version 17 of net.sf.basedb.opengrid/using


Ignore:
Timestamp:
Aug 21, 2020, 9:25:10 AM (4 years ago)
Author:
Nicklas Nordborg
Comment:

Added information about Slurm in first chapter

Legend:

Unmodified
Added
Removed
Modified

  • net.sf.basedb.opengrid/using

    v16 v17  
    1 = How to use the Open Grid Scheduler package API =
    2 In this tutorial we will try to describe the main aspects of the programmatic API that other extensions can use in order to access and use Open Grid Clusters.
     1= How to use the Job scheduler package API =
     2In this tutorial we will try to describe the main aspects of the programmatic API that other extensions can use in order to access and use Open Grid and Slurm clusters.
    33
    44[[PageOutline(2-3,,inline,numbered)]]
    … …  
    88== The use case ==
    99
    10 The use case in this tutorial is that we are going to implement an extension that wants to display a web page inside BASE for starting a job on an Open Grid Cluster. We focus on the Open Grid part of this and care less about how the extension interacts with BASE. You may want to read more about this in the [http://base.thep.lu.se/chrome/site/latest/html/developer/extensions/index.html BASE documentation]. We assume that you have already implemented an extension that can display a web page that allows the user to specify input parameters for the job. On this web page you also want the user to be able to select an Open Grid Cluster that the job should be submitted to. The list below is an overview of the main steps that are needed.
    11 
    12  1. Display a selection list with available Open Grid Clusters
     10The use case in this tutorial is that we are going to implement an extension that wants to display a web page inside BASE for starting a job on a cluster. We focus on the Job scheduler part of this and care less about how the extension interacts with BASE. You may want to read more about this in the [http://base.thep.lu.se/chrome/site/latest/html/developer/extensions/index.html BASE documentation]. We assume that you have already implemented an extension that can display a web page that allows the user to specify input parameters for the job. On this web page you also want the user to be able to select a cluster that the job should be submitted to. The list below is an overview of the main steps that are needed.
     11
     12 1. Display a selection list with available clusters
    1313 2. Submit the job parameters and other information to a servlet
    1414 3. The servlet creates a job script and submit it to the cluster
    1515 4. Get a notification once that job has been completed, so that we can import files and other data back into BASE
    1616
    17 == Enumerating Open Grid Clusters ==
    18 
    19 In this step we want to display a selection list on the web page that allows the user to select a pre-defined Open Grid Cluster. We recommend that the list is populated by !JavaScript code that uses AJAX, a Servlet and JSON to retreive the information. You need to implement everything on the browser side and most of the servlet in your own extension package.
     17**Note! ** The API has a lot of classes and methods that have `OpenGrid` in the name. This doesn't mean that the API is only usable for Open Grid clusters. The same API can also be used for Slurm clusters. The naming is only for historical reasons and in order to maintain backwards compatibility.
     18
     19== Enumerating clusters ==
     20
     21In this step we want to display a selection list on the web page that allows the user to select a pre-defined cluster. We recommend that the list is populated by !JavaScript code that uses AJAX, a Servlet and JSON to retreive the information. You need to implement everything on the browser side and most of the servlet in your own extension package.
    2022
    2123The `OpenGridService` class is typically the starting point for a lot of actions. From this class it is possible to get information about and access all clusters that have been defined in the `opengrid-config.xml` file. The service is a singleton, use the `OpenGridService.getInstance()` method to get the instance. Note! It is important that the service is actually running inside BASE. Check the '''Administrate->Services''' page that this is the case, otherwise you will get an empty service.
    2224
    23 To enumerate the available Open Grid Clusters use one of the `OpenGridService.getClusters()` methods. This will return a collection of `OpenGridCluster` instances. Most methods in this class are used for getting configuration information from the `opengrid-config.xml` file. The `OpenGridCluster.getId()` method returns the internal ID of the cluster definition. It is created by combining the username, address and port of the cluster (for example, `griduser@grid.example.com:22`). The ID can then be used with `OpenGridService.getClusterById()` to directly access the same cluster later on. Other useful information can be found in the objects returned by calling `OpenGridCluster.getConnectionInfo()` and `OpenGridCluster.getConfig()`. The `OpenGridCluster.asJSONObject()` contains more or less the same information wrapped up as JSON data. This is useful for transferring information a web interface to allow a user to select a cluster to work with.
     25To enumerate the available clusters use one of the `OpenGridService.getClusters()` methods. It is possible to return all configured cluster or only a subset defined by a filter. All methods will return a collection of `OpenGridCluster` instances. Most methods in this class are used for getting configuration information from the `opengrid-config.xml` file. The `OpenGridCluster.getId()` method returns the internal ID of the cluster definition. It is created by combining the username, address and port of the cluster (for example, `griduser@grid.example.com:22`). The ID can then be used with `OpenGridService.getClusterById()` to directly access the same cluster later on. Other useful information can be found in the objects returned by calling `OpenGridCluster.getConnectionInfo()` and `OpenGridCluster.getConfig()`. The `OpenGridCluster.asJSONObject()` contains more or less the same information wrapped up as JSON data. This is useful for transferring information a web interface to allow a user to select a cluster to work with.
    2426
    2527'''Java code in a servlet running on the BASE web server'''
    … …  
    3537JSONArray jsonHosts = new JSONArray();
    3638
    37 // Enumerates all clusters that the current user has access to
     39// Enumerates all clusters that the current user has access to (no filtering)
    3840for (OpenGridCluster host : service.getClusters(dc, Include.ALL))
    3941{
    … …  
    5658{
    5759   var cluster = clusters[i];
    58    var option = new Option(cluster.connection.name, cluster.id);
     60   var option = new Option(cluster.connection.name + ' ('+cluster.config.type+')', cluster.id);
    5961   option.cluster = cluster;
    6062   list[list.length] = option;
    … …  
    6365
    6466Note that there is no need to use the `OpenGridCluster.connect()` method yet.
     67
     68Some of the `OpenGridService.getClusters()` methods can take a filter parameter. This makes it possible
     69to only list clusters with some specific properties. Predefined filter implementations can be found in the `net.sf.basedb.opengrid.filter` package:
     70
     71 * `xxx`: Filter on type of cluster (eg. Slurm or Open Grid)
     72 * `xxx`: Only return clusters that we can connect to
     73 * `xxx`: Only return clusters that we connect to with a given username
    6574
    6675== Creating a job script ==