GA ListSending Engine API V3
This is the API Documentation, Version 3 for the GA Listsend System
Here is the messaging API
HTTP Submission API
- Table of Contents
- Overview
- Example
- Compression
- Request Document
- Request for Multiple Messages
- HTTP keep-alive
- Message Acceptance Throttling and Back-Pressure
- Logging API Calls
Overview
GA Listsend provides an HTTP message submission API.
The information is provided to the API as a POST or PUT request of JSON content. (The header of content-type: application/json
must be provided.)
The API provides a response of JSON content with a content type of application/json
.
The uploaded information can be compressed using gzip
or deflate
compression. The header of Content-Encoding: gzip
or Content-Encoding: deflate
must be provided when doing this.
There is a 10MB size limit per submission. This limit is applied before a compressed payload is uncompressed, so if, for example, a 25MB payload compresses down to 8MB, you can submit it, but if it compresses down to 11MB, you cannot.
Authentication is performed by providing a username/password for an Email User in the JSON document.
The API is provided at /api/v1/send.json
on your GreenArrow server’s HTTP service.
In order to maximize your injection speed, we strongly recommend use of batching (see the Request for Multiple Messages section of this document) and HTTP keep-alive.
Example
Example JSON document:
{
"username":"test@example.com",
"password":"test",
"message":{
"html":"html content goes <b>here</b>",
"text":"text content goes here",
"subject":"this is the subject",
"to":[
{
"email":"recipient@example.com",
"name":"John Doe"
}
],
"from_email":"from@example.com",
"from_name":"Your Company"
}
}
Example JSON reply for success:
{"success":1,"message_id":"mid-990f39eb71e52cc5050b7b36ada937d4@example.com"}
Example JSON reply for failure:
{"success":0,"error":"no data in POST or PUT payload"}
or:
{"success":0,"error":"incorrect username\/password"}
Submitting this request with curl
:
cat <<'EOT' > post_body.txt
{
"username":"test@example.com",
"password":"test",
"message":{
"html":"html content goes <b>here</b>",
"text":"text content goes here",
"subject":"this is the subject",
"to":[
{
"email":"recipient@example.com",
"name":"John Doe"
}
],
"from_email":"from@example.com",
"from_name":"Your Company",
"mailclass":"trans",
"headers":{
"X-foo":"bar"
}
}
}
EOT
curl -X POST -H "Content-Type: application/json" --data-binary @post_body.txt http://127.0.0.1/api/v1/send.json
The API can also be accessed through TLS:
(Remove the --insecure
option if you have a valid TLS certificate.)
curl -X POST -H "Content-Type: application/json" --data-binary @post_body.txt --insecure https://127.0.0.1/api/v1/send.json
Compression
The content can be gzip
or deflate
compressed. This allows payloads with lots of messages to be reduced in size.
This requires the Content-Encoding: gzip
or Content-Encoding: deflate
header on the HTTP request.
Example:
cat <<'EOT' | gzip -c -9 > post_body.txt.gz
{
"username":"test@example.com",
"password":"test",
"message":{
"html":"html content goes <b>here</b>",
"text":"text content goes here",
"subject":"this is the subject",
"to":[
{
"email":"recipient@example.com",
"name":"John Doe"
}
],
"from_email":"from@example.com",
"from_name":"Your Company",
"mailclass":"trans",
"headers":{
"X-foo":"bar"
}
}
}
EOT
curl -X POST -H "Content-Type: application/json" -H "Content-Encoding: gzip" --data-binary @post_body.txt.gz http://127.0.0.1/api/v1/send.json
Request Document
username
- email address of an email user (NOT the username of GreenArrow web-interface login)password
- password of above usermessage
- Object containing the following fields:
id string /optional |
Extra identifier for the message. This value is returned in the response in order to assist in mapping specific messages to responses. It is not otherwise used. It is not the same as the |
||||||||
html string |
HTML version of the content for the email message. Either the |
||||||||
text string |
Text version of the content for the email message. Either the |
||||||||
amp_html string |
AMP for Email version of the content for the email message. This cannot be the only part of the email (i.e. you must specify |
||||||||
subject string /required |
|
||||||||
envelope_recipients array /optional |
The list of envelope recipients to which to deliver this email. Envelope recipients are the recipients, separate from the recipients specified in the email headers, to which the email will actually be delivered. They are the RFC5321.RcptTo addresses and are transmitted using the SMTP If not specified (meaning the Each entry in the list is a string representing an email address. If the domain name of the email address contains UTF-8 characters, it must be converted to Punycode prior to calling this API. |
||||||||
to array /required List of To recipients. These are used to create the Each entry in the list is an object consisting of:
|
|||||||||
cc array /optional List of CC (carbon copy) recipients. These are used to create the Each entry in the list is an object consisting of:
|
|||||||||
bcc array /optional |
List of BCC (blind carbon copy) recipients. If Each entry in the array is a recipient’s email address. If the domain name of the email address contains UTF-8 characters, it must be converted to Punycode prior to calling this API. |
||||||||
from_email string /required |
Email address to use in |
||||||||
from_name string /optional |
Name to use in |
||||||||
return_path string /optional |
Return-Path to use. This is also known as the bounce address and envelope sender. Note that this gets overwritten if SimpleMH is configured to set the Return-Path for you. |
||||||||
mailclass string /default: The default Mail Class |
Name of the SimpleMH Mail Class to use. This key has the same effect as the |
||||||||
message_id_domain string /optional |
Fully qualified domain name to use in the domain portion of the |
||||||||
headers hash /optional |
Headers to add to the email. The key is the header name and the value is the raw text of the value of the header. If either the The automatic Date header takes the form:
The automatic Message-ID header takes the form:
See the |
||||||||
attachments array /optional Attachments to add to the message. Each element in the array is an object consisting of:
|
Request for Multiple Messages
Up to 500 messages may be submitted per API request.
(Note: Not all messages in a batch are guaranteed to be queued. A response will be provided within 31 seconds with as many messages as could be queued in that time. See the section “Message Acceptance Throttling and Back-Pressure” below.)
To submit multiple messages in a single request, provide a document like this:
{
"username":"test@example.com",
"password":"test",
"messages":[
{
"html":"html content goes <b>here</b>",
"text":"text content goes here",
"subject":"this is the subject",
"to":[
{
"email":"recipient-1@example.com",
"name":"John Doe"
}
],
"from_email":"from@example.com",
"from_name":"Your Company"
},
{
"html":"html content goes <b>here</b>",
"text":"text content goes here",
"subject":"this is the subject",
"to":[
{
"email":"recipient-2@example.com",
"name":"Bob Smith"
}
],
"from_email":"from@example.com",
"from_name":"Your Company"
}
]
}
The JSON document most be an object with the following keys:
Key | Description |
---|---|
username |
Same as single message document |
password |
Same as single message document |
messages |
A list containing one or more objects which contain the same information as the message key in a single message document. |
The output can be in two formats.
(1) An error that applies globally to the entire request:
{"success":0,"error":"incorrect username\/password"}
(2) Or the overall request succeeded and a specific success value is provided for each message:
{
"success":1,
"messages":[
{
"success":1,
"message_id":"mid-90df674d4c98c1aa647eaca724086ef0@example.com",
"attempted":1,
"id":"1"
},
{
"success":1,
"message_id":"mid-e20664ffd71475b270b586889872e4f7@example.com",
"attempted":1,
"id":"2"
}
]
}
If there is an internal error, then the subsequent messages in the request will not be attempted. For example:
{
"success":1,
"messages":[
{
"success":1,
"message_id":"mid-90df674d4c98c1aa647eaca724086ef0@example.com",
"attempted":1,
"id":"1"
},
{
"success":0,
"error":"internal error: unable to make socket connection (Connection refused)",
"attempted":1,
"id":"2"
},
{
"success":0,
"error":"not attempting due to previous internal errors",
"attempted":0,
"id":"3"
}
]
}
In this request of three messages, the second message encountered an internal error so the third message was not even attempted. The attempted
key provides data on if the message was even attempted.
It is very important to handle error conditions and retry messages appropriately.
If the attempted
value is 0
for a message, this lets you know that you should retry it.
Here are the two error messages that cause messages to not be attempted:
-
not attempting due to previous internal errors
– a previous message encountered an internal error, so the rest of the messages in the batch are not attempted and return this error message. Please retry these messages in a different batch. -
not attempting because previous messages have taken too long
– this happens when the system is not able to pre-process messages as fast as you are trying to send them to this API (see “Throttling” below). Only the messages that can be accepted within about 30 seconds will be accepted, and the rest of the messages in the batch will not be attempted and return this error. This is put in place so that an HTTP timeout does not prevent you from getting any answer to the HTTP request.
HTTP keep-alive
For maximum speed, it is recommended to use HTTP keep-alive in your HTTP API, so that you re-use the same TCP connection for each new request.
This can have a significant performance impact because TCP slow-start is bypassed for subsequent requests.
Message Acceptance Throttling and Back-Pressure
Message batches may not be larger than 500 messages.
However, not all messages in a batch are guaranteed to be queued.
The HTTP API will queue as many messages as possible within 31 seconds and then provide a status response indicating which messages were queued. (This 31 seconds does not include network delay for transferring data.) The purpose of replying within 31 seconds is to prevent HTTP timeouts – which would prevent the caller from knowing whether messages were successfully queued or not.
This is how GreenArrow throttles how fast messages are accepted through the HTTP API so that they can not be accepted faster than messages are pre-processed (click/open tracking & DKIM signing) and added to the MTA’s ram-queue. This is a form of “back-pressure,” which is the MTA slowing down the message injection to the speed it can handle.
(Another common form of “back-pressure” is a fraction-of-a-second delay on accepting a message in an SMTP conversation, which has the effect of slowing down the overall speed of messages injection.)
Messages that were unable to be queued due to this limit will be returned with the not attempting because previous messages have taken too long
error message (documented above). It is imperative that you re-submit messages returned with this (and the other) error messages documented above.
This error message is the MTA communicating that the batch size is too large.
Recommended batch size
We recommend that your batches be small enough to be accepted within 10 seconds.
There is little benefit to batches that take longer than 10 seconds to be accepted. (At this size, the per-batch overhead is a very small portion of the equation, and larger batches do not gain much if anything.)
There is no benefit (and only a cost) to attempting batches larger than can be handled within 31 seconds.
How many messages can be processed within 10 seconds depends on many things, for example: how many other senders are concurrently using the HTTP API, how many messages are currently being injected through other injection methods, and the overall capacity of the system.
We recommend either (a) using a conservative static batch size or (b) dynamically adjusting the batch size based on injection speed.
To dynamically adjust the batch size:
- for the last batch, divide how many messages were accepted by how long the batch took, to calculate the messages per second speed.
- multiply the messages per second speed by 10 to get the size of the next batch
- if larger than 500, reduce to 500
How to observe back-pressure
If message acceptance is currently being throttled due to back-pressure, the size of the Simplemh processing queue
in the hvmail_status status
output will be above 90%.
Here is an example:
Simplemh processing queue: 98% used ( 1960/ 2000) (messages)
99% used ( 99/ 100) (MB of memory)
Logging API Calls
GreenArrow can write details about HTTP Submission API requests to the Apache error log. To enable writing details to Apache error log, create the file /var/hvmail/control/opt.send_api_debug_log
. For example:
echo 1 > /var/hvmail/control/opt.send_api_debug_log
Remove the file to disable logging to Apache’s error log:
rm -f /var/hvmail/control/opt.send_api_debug_log
Apache error log is saved to /var/hvmail/apache/logs/error_log
. Here is an example of what GreenArrow logs to Apache’s error log (JOSN objects formatted to improve readability):
[Tue Jun 29 08:08:19.707090 2021] [php7:notice] [pid 19557] [client 1.2.3.4:51269]
http_send_api:
input={
"username": "test@example.com",
"password": "test",
"message": {
"html": "html content goes <b>here</b>",
"text": "text content goes here",
"subject": "this is the subject",
"to": [
{
"email": "recipient@example.com",
"name": "John Doe"
}
],
"from_email": "from@example.com",
"from_name": "Your Company",
"mailclass": "trans",
"return_path": "bounces@example.com"
}
}
output={
"success": 0,
"error": "incorrect username/password"
}
input=
is the JSON object sent to GreenArrow by the API client.output=
is the JSON object in the response from GreenArrow.
Here is the Stats API
Stats API
- Table of Contents
- Overview
- Index of Campaigns and ListIDs
- Campaign Stats
- Recipient Stats
Overview
GA ListSend provides an API for retrieving statistics in embeddable HTML format, or a JSON or PHP serialized array. Stats can be retrieved for campaigns or recipients. This allows you to directly embed GA Listsend's reports into your own application.
Authenticate your request using HTTP Basic authentication with the username and password that you use to access the GA Listsend's administration interface.
Index of Campaigns and ListIDs
This section describes how to make an API call which returns indexes of Campaigns and ListIDs similar to what’s seen when clicking on the Statistics
tab within GA Listsend's web interface.
Using the API
Make an HTTP GET request to /greenarrowadmin/h/stats_sends_list.php
on your server.
Add the following GET query parameters:
Key | Value | Required | Description |
---|---|---|---|
api |
string | Yes | Format of the the report. The following options are available: • json - raw data formatted as a JSON serialized array• php - a serialized PHP array• print_r - an array that’s been processed by PHP’s print_r function |
listid |
string | No | The ListID that you wish to limit results to. If this parameter is excluded, the API will include all ListIDs in its results |
_page_size |
integer | No | The maximum number of results to display. The default value is 100 |
_page |
integer | No | The page number to display in the results. Counting starts from 0 , and by default, the first page is displayed. |
injtime_first |
string | Required if injtime_first_start and injtime_first_end are set |
The date range that you wish to display results for.This field should be set equal to custom if injtime_first__start and injtime_first__end are set. Otherwise it is optional. Value values are:• all - All dates• last_7 - Last 7 days• last_14 - Last 14 days• last_30 - Last 30 days. This is the default value• last_60 - Last 60 days• last_90 - Last 90 days• custom - Custom range defined by injtime_first__start and injtime_first__end |
__changeform |
integer | Required if injtime_first_start and injtime_first_end are set |
This field should be set equal to 1 if the injtime_first__start and injtime_first__end values are set. Otherwise it should be excluded. |
injtime_first__start |
string | Required if injtime_first__end is set |
The start of the date range that you wish to display results for in mm/dd/yyyy format. For example, to use a start date of January 1, 2013, you would set this value to 01/01/2013 , which would be URL encoded as 01%2F01%2F2013 . |
injtime_first__end |
string | Required if injtime_first__start is set |
The end of the date range that you wish to display results for in mm/dd/yyyy format. For example, to use an end date of December 31, 2013, you would set this value to 12/31/2013 , which would be URL encoded as 12%2F31%2F2013 . |
The date based parameters listed above (injtime_first
, injtime_first__start
, and injtime_first__end
) all filter results in reference to the date that GA Listsend received the first message in a campaign. For example, if a campaign started sending in January 1st, and completed on January 2nd, the date that GA Listsend would use for that campaign in its filtering criteria would be January 1st.
Data Returned After a Successful API Call
Key | Value | Description |
---|---|---|
success |
integer | 1 if success, 0 if failure |
mailing_lists |
array | An array containing one element for each mailing list for which there are stats available. Each element’s key is the list’s ListID. If GA Listsend has a more user friendly description for the mailing list, such as the GreenArrow Studio mailing list name, that description goes into that element’s value. Otherwise the value contains the ListID again. |
sends |
array | Any array containing one element for each “Send”, or “Campaign” that GA Listsend has stats for which meet the filtering criteria. This array is described in more detail in the next table. |
The structure of the sends
array that is included in API call results is described below:
Key | Value | Description |
---|---|---|
id |
integer | The SendDBID of the campaign. This is not the same thing as the SendID. See this page’s “SendID and SendDBID” section for more details. |
sendid |
string | The SendID of the campaign. |
injtime_first |
integer | The time, in seconds since the UNIX epoch that GA Listsend received the first message for this campaign. |
injtime_last |
integer | The time, in seconds since the UNIX epoch that GA Listsend received the last message for this campaign. |
total_messages |
integer | The number of messages in this campaign |
total_failure |
integer | The number of bounces in this campaign |
total_inqueue |
integer | The number of messages still in queue for this campaign |
lists_str |
string | This field is not currently documented. |
Data Returned After a Failed API Call
Key | Value | Description |
---|---|---|
success |
integer | 1 if success, 0 if failure |
error_text |
string | Text message for this error. |
Example
URL
The following URL corresponds to any API call that specifies that the results should be processed by PHP’s print_r
function before being returned:
http://mailer.example.com/greenarrowadmin/h/stats_sends_list.php?api=print_r
Result
Here are example results:
Array
(
[success] => 1
[mailing_lists] => Array
(
[47] => demo
[t23] => t23
)
[sends] => Array
(
[0] => Array
(
[id] => 991
[sendid] => 131
[injtime_first] => 1381967540
[injtime_last] => 1381967972
[total_messages] => 59704
[total_failure] => 541
[total_inqueue] => 0
[lists_str] => 47,59704
)
[1] => Array
(
[id] => 990
[sendid] => 130
[injtime_first] => 1381967152
[injtime_last] => 1381967152
[total_messages] => 1
[total_failure] => 0
[total_inqueue] => 0
[lists_str] => t23,1
)
)
)
SendID and SendDBID
You may have noticed that when viewing the stats for a send in GA Listsend, part of the URL contains a SendDBID. For example:
http://mailer.example.com/greenarrowadmin/h/stats_sends_view.php?senddbid=692
Even though SendDBID sounds a lot like SendID, it’s not the same thing. The SendID is what you’re working with when interacting with GA Listsend. The SendDBID is value that GA Listsend uses internally to uniquely identify sends. SendIDs are described in the GA Concepts page.
Campaign Stats
Using the API
Make a request to /greenarrowadmin/h/stats_sends_view.php
on your server.
Add the following GET query parameters:
Key | Value | Description |
---|---|---|
sendid |
string | The SendID that you want to get data for |
api |
string | Format of the the report Normally the api key would be assigned a value of embed_html to retrieve embeddable HTML. There are additional options available though, if you prefer to work with raw data:• embed_html - Return embeddable HTML. This is what is normally used.• json - raw data formatted as a JSON serialized array• php - a serialized PHP array• print_r - an array that’s been processed by PHP’s print_r function |
The remainder of this section assumes that the api
key was assigned a value of embed_html
.
The data returned is a JSON serialized array. The value of the head
key in this array should be placed within the <head>
tags of your web page. The value of the html
key should be placed within the <body>
tags of your web page.
We also recommend setting these styles, if your CSS does not already do this:
<style type="text/css">
body { font-size: 12px; }
table { font-size: 12px; }
</style>
Example
The following example retrieves the Send Statistics for a campaign with a SendID of 4
:
PHP Code
<?php
$url = "http://admin:PASSWORD@hostname/greenarrowadmin/h/stats_sends_view.php?sendid=4&api=embed_html";
$data = file_get_contents($url);
if ( substr($data, 0, 2) != '{"' )
die("error");
$array = json_decode($data, true);
?>
<html>
<head>
<style type="text/css">
body { font-size: 12px; }
table { font-size: 12px; }
</style>
<?php echo $array['head']; ?>
</head>
<body>
<div style="width: 870px; padding: 20px; ">
<?php echo $array['html']; ?>
</div>
</body>
</html>
Result
Recipient Stats
Using the API
Make a request to /greenarrowadmin/h/stats_sends_view_logs.php
on your server.
Add the following GET query parameters:
Key | Value | Description |
---|---|---|
sendid |
string | The SendID that you want to get data for |
api |
embed_recipient_status_html | Retrieves embeddable HTML |
email |
string | Recipient email address that you want to get data for |
The data returned is a JSON serialized array. The value of the head
key in this array should be placed within the <head>
tags of your web page. The value of the html
key should be placed within the <body>
tags of your web page.
We also recommend setting these styles, if your CSS does not already do this:
<style type="text/css">
body { font-size: 12px; font-family: Arial, sans-serif; }
table { font-size: 12px; font-family: Arial, sans-serif; }
table.quick_data {
border-collapse:collapse;
border-spacing:0;
}
table.quick_data tr td {
border: 1px solid;
padding: 3px;
text-align: left;
vertical-align: top;
margin: 0;
}
</style>
Example
The following example retrieves the Send Statistics for the test@example.com recipient for a campaign with a SendID of 4
:
PHP Code
<?php
$url = "http://admin:PASSWORD@mailer.example.com/greenarrowadmin/h/stats_sends_view_logs.php?sendid=4&api=embed_recipient_status_html&email=test@example.com";
$data = file_get_contents($url);
if ( substr($data, 0, 2) != '{"' )
die("error");
$array = json_decode($data, true);
?>
<html>
<head>
<style type="text/css">
body { font-size: 12px; font-family: Arial, sans-serif; }
table { font-size: 12px; font-family: Arial, sans-serif; }
table.quick_data {
border-collapse:collapse;
border-spacing:0;
}
table.quick_data tr td {
border: 1px solid;
padding: 3px;
text-align: left;
vertical-align: top;
margin: 0;
}
</style>
<?php echo $array['head']; ?>
</head>
<body>
<div style="width: 870px; padding: 20px; ">
<?php echo $array['html']; ?>
</div>
</body>
</html>
Result
The following is a link to the overall system API:
https://www.greenarrowemail.com/docs/greenarrow-engine/API-V3/