Announcing Mantl API

October 29, 2015 // Ryan Eschinger // Mantl

Mantl API provides a new way for you to manage Mantl clusters. With the first release, you can easily install pre-built applications and Mesos frameworks. With a single API call, you can now spin up Cassandra on your Mantl cluster.

Mesosphere and its partners have been working hard building packages for various Mesos frameworks, databases, and other software applications. While these packages can be used with Mantl, it is often more difficult than it should be due to underlying differences between Mantl and the Datacenter Operating System (DCOS). There needs to be a simple way to use the great packages that are available. And, it needs to be as flexible as possible — you should be able to install packages via automation scripts, command line tools, or web interfaces. Mantl API makes this possible.

At this time, Mantl API supports the ability to view, install, and uninstall Mesosphere packages on Mantl. It relies on the mesosphere universe package repository as well as a Mantl specific package repository. The packages in mantl-universe can be installed on Mantl right now. You can also attempt to install any of the Mesosphere packages through the API but they haven’t been tested yet and you will likely have to make some configuration changes before they will work. It would be great to see some contributions to get more packages supported on Mantl.

Deploying Mantl API

You can run Mantl API on top of Marathon. Save the JSON below as a file called mantl-api.json:

{
  "id": "/mantl-api",
  "container": {
    "type": "DOCKER",
    "docker": {
      "image": "ciscocloud/mantl-api:0.1.0",
      "network": "BRIDGE",
      "portMappings": [
        { "containerPort": 4001, "hostPort": 0 }
      ],
      "forcePullImage": true
    }
  },
  "instances": 1,
  "cpus": 1.0,
  "mem": 512,
  "constraints": [["hostname", "UNIQUE"]],
  "env": {
    "CONSUL_HTTP_SSL_VERIFY": "false",
    "MANTL_API_LOG_LEVEL": "debug",
    "MANTL_API_CONSUL": "https://consul.service.consul:8500",
    "MANTL_API_MARATHON": "https://marathon.service.consul:8080",
    "MANTL_API_MARATHON_USER": "admin",
    "MANTL_API_MARATHON_PASSWORD": "yourpw",
    "MANTL_API_MARATHON_NO_VERIFY_SSL": "true",
    "MANTL_API_MESOS_PRINCIPAL": "mantl-api",
    "MANTL_API_MESOS_SECRET": "yoursecret"
  },
  "healthChecks": [
    {
      "protocol": "HTTP",
      "path": "/health",
      "gracePeriodSeconds": 5,
      "intervalSeconds": 10,
      "portIndex": 0,
      "timeoutSeconds": 10,
      "maxConsecutiveFailures": 5
    }
  ]
}

You will need to update MANTL_API_MESOS_SECRET with a valid Mesos credential for your cluster. This can be found in your security.yml file — see the Mantl security-setup documentation for more information. You will also need to update MANTL_API_MARATHON_USER and MANTL_API_MARATHON_PASSWORD with your Marathon credentials (see marathon_http_credentials in your security.yml file).

Note: If you built your Mantl cluster with Mesos and Marathon authentication disabled, you can adjust mantl-api.json by removing the MANTL_API_MARATHON_USER, MANTL_API_MARATHON_PASSWORD, MANTL_API_MESOS_PRINCIPAL, and MANTL_API_MESOS_SECRET variables. If you also turned off SSL, you can remove MANTL_API_MARATHON_NO_VERIFY_SSL and you should change https to http in the MANTL_API_CONSUL and MANTL_API_MARATHON variables.

After updating the necessary credentials, the above JSON should be enough to run the API on Mantl clusters with a default configuration. If you had customized some of your cluster configuration, such as the Consul domain name, you may have to adjust the variables in env accordingly. You can see the full list of options that Mantl API supports in the Mantl API documentation.

Now, we can submit this to Marathon:

curl -ku admin:mantlpw -X POST -H "Content-type: application/json" -d @mantl-api.json https://mantl-control-01/marathon/v2/apps

You will need to update admin:mantlpw with the same Marathon credentials you included in the JSON above. You will also need to replace mantl-control-01 with a valid host name of one of your control nodes.

After a few minutes, you should see mantl-api running and healthy in Marathon.

Healthy Mantl API (mantl-api) in Marathon

Installing a Package

Before we can talk to the Mantl API, we need to discover the host on which Marathon launched the task and its port. We can get this through the Marathon API with a command like the following:

curl -sku admin:mantlpw https://mantl-control-01/marathon/v2/apps/mantl-api | jq ".app.tasks[] | .host, .ports"
"mantl-worker-003"
[
  4483
]

This command asks the Marathon API for the Mantl API app configuration and status. It uses jq (an awesome tool for processing JSON at the command line) to parse the list of tasks and outputs the hosts and ports where they are running. If you are following along, there will be one task running on a single host and listening on a single port. In this example, Mantl API is running on mantl-worker-003 on port 4483.

You can also find this information via the Marathon UI or in Consul.

If you are working outside of the Mantl cluster network, you are going to need to ensure that you are able to reach the API port. You will to update the security groups or firewall rules for your platform to allow access to the API port from your own network.

Now that we know how to reach Mantl API, we can install packages with a single API call. Let’s install Cassandra on our cluster.

curl -X POST -d "{\"name\": \"cassandra\"}" http://mantl-worker-003:4483/1/packages

It can take 5-7 minutes for the Cassandra framework to become healthy. When it does, you will see it running in Marathon and you will see various tasks running in Mesos across your cluster.

Uninstalling a Package

Uninstalling a package is simple too.

curl -X DELETE http://mantl-worker-003:4483/1/packages/cassandra

This will shutdown the Mesos framework, destroy the application in Marathon, and clean up the framework state from Zookeeper.

Getting Information About Packages

You can get a list of the available packages by running the following command:

curl http://mantl-worker-003:4483/1/packages

This will return a list of packages along with their metadata. One important property is the supported flag. If this is true, it means that it should install without customization on Mantl. You can get a list of the supported packages by running the following command:

curl -s http://mantl-worker-003:4483/1/packages | jq ".[] | select(.supported==true) | .name"
"cassandra"
"hdfs"
"kafka"
"memsql"
"riak"

If you want to see all of the metadata associated with a specific package, you can run:

curl -s http://mantl-worker-003:4483/1/packages/hdfs | jq .
{
  "name": "hdfs",
  "description": "Hadoop Distributed File System (HDFS), Highly Available",
  "framework": true,
  "currentVersion": "0.1.4",
  "supported": true,
  "tags": [
    "mesosphere",
    "framework",
    "filesystem"
  ],
  "versions": {
    "0.1.3": {
      "version": "0.1.3",
      "index": "0",
      "supported": false
    },
    "0.1.4": {
      "version": "0.1.4",
      "index": "1",
      "supported": true
    }
  }
}

How does Mantl API compare to the DCOS CLI?

At this time, the only overlap between the DCOS CLI and Mantl API is the ability to install packages. While it is possible to use the DCOS CLI against a Mantl cluster, the functionality is limited. And, it is not compatible with some of the on-by-default Mantl security features such as SSL and authentication.

Mantl API and DCOS CLI both use the Mesosphere Universe package repository but Mantl API stores it in the Consul key/value store. It layers the mantl-universe repository on top in order to provide the Mantl-specific customizations. There are a couple of other differences between the two tools: Mantl API is written in Go and is an HTTP-based API rather than a command line tool.