On this page

Warp 10

Main concepts

Warp 10 is a time series database. The notion of class, labels, longitude, latitude, altitude and value are used.

A GTS is a GeoTime Serie defined by a class and somes labels. They are indexed and used to quickly retrieved the data.

labels is a kind of dictionnary. That is called a map under the warp10 terminology.

A GTS may contain some values which have the following models : [ timestamp longitude latitude altitude value ]

Warp 10 uses Warp Script. It’s a stack based language using reverse polish notation.

The metrics ovh team built an other way to request Warp 10: https://github.com/ovh/tsl/

Warp 1O documentation is availlable on their website

Time and duration in Warp 10

More information about date functions here, and time functions here.

Note:

The plateform's time unit is in microsecond.

Duration is set by the followings symbols

  • d : day
  • h : Hour
  • ms : millisecond
  • ns : nanosecond
  • ps : picosecond
  • s : second
  • us : microsecond
  • w : week

Date

Allowed format :

  • ISO8601
  • Timestamp in microsecond

Builtin function :

  • NOW : get the current timestamp
  • ISO8601 : Convert a string or a timstamp to a iso8601 date format

Endpoint

The Clever Cloud Warp 10 endpoint is:

https://c1-warp10-clevercloud-customers.services.clever-cloud.com/api/v0

You can find documentation about endpoint gateway here.

You can find the endpoint and an available token under the metric tab of your application

You can query our Warp 10 platform with your own script. Here’s a curl example :

  curl -T <Path/to/a/warpscript_file> https://c1-warp10-clevercloud-customers.services.clever-cloud.com/api/v0/exec

Do not forget the endpoint. exec in the previous exemple.

Token

Tokens are based on your application with the notion of producer and owner. Hence, only the data owner can see it.

You can find a 5 days available token in the metric tab of your application.

Technical constraints

The followings limits are defined in Warp 10. The soft one can be passed over by an AUTHENTICATE operation. The hard one is unsurpassable.

WarpScript OperatorWarp 10 limit descriptionsoft limithard limit
MAXGTSMaximum number of GTS which can be fetched10e55e7
LIMITmaximum number of datapoints which can be fetched during a script execution10e610e7
MAXBUCKETSmaximum number of buckets which can be created by a call to BUCKETIZE10e550e5
MAXDEPTHmaximum depth (number of levels) of the execution stack5e35e3
MAXLOOPmaximum number of milliseconds which can be spent in a loop5e310e3
MAXOPSmaximum number of operations which can be performed during a single WarpScript execution5e65e7
MAXSYMBOLSmaximum number of simultaneous symbols which can be defined on the stack during a single WarpScript execution64256
MAXGEOCELLSmaximum number of cells a GEOSHAPE10e310e4
MAXPIXELSmaximum size (in pixels) of images which can be created by PGraphics10e510e5
MAXRECURSIONmaximum nesting depth of macro calls1632

Warning:

operation over soft limts may be really intensives.

Usage:

An example where it is needed to increase the fetch limit by the LIMIT function

'<READTOKEN>' AUTHENTICATE
50e6 TOLONG LIMIT
// Fetch on the 'accessLogs' class for your application id as labels
[ '<READTOKEN>' 'accessLogs' { 'app_id' '<APP_ID>'  } NOW 1 w ] FETCH

Visualization and exploration

Quantum

Quantum is a web tool used to run some WarpScript. You can access to it from your metrics interface.

It provide the path to the Clever Cloud Warp 10 gateway and let you explore your data.

Macro

Warp 10 provide a server side macro manager. It is a way to release some ready to use WarpScript. Hence, Clever Cloud provides some macros as helpers to avoid redondant and often need code.

Note:

More information on the Warp 10' macros documentations.

Consumption

The following macros are helpers to compute consumption in seconds

  • app_consumption

Return the consumption in second by applications for a specific organisation. Start and End parameters can be either a timestamp in microseconds or an iso8601 date format.

'<READ TOKEN>' '<ORGANISATION ID>' '<START>' '<END>' @clevercloud/app_consumption
  • orga_consumption

Return all the consumption in second for a specific organisation. Start and End parameters can be either a timestamp in microseconds or an iso8601 date format.

'<READ TOKEN>' '<ORGANISATION ID>' '<START>' '<END>' @clevercloud/app_consumption

AccessLogs

  • fetch_accessLogs_key_v0

We provide the following macro to easily and quickly deep dive into access logs data. As we store access log as a Json value in a geotime series (metrics documentation here), this macro can be useful for straightforward access to a specific key. it allows you to fetch the accessLogs class and get only wanted value instead of the whole Json.

  '<READ TOKEN>' { 'app_id'  'id' } '<1stLevelKey>' NOW 1 h  @clevercloud/fetch_accessLogs_key_v0

  '<READ TOKEN>' { 'app_id'  'id' } '<1stLevelKey>.<2ndLevelKey>' NOW 10 m  @clevercloud/fetch_accessLogs_key_v0

More example in the metrics part of this documenation.

Nested keys can be reached using a dot (.) to represent the depth.

'<READ TOKEN>' '<LABELS>' '<KEY>' '<START>' '<END>' @clevercloud/fetch_accessLogs_key_v0

Comments