Deleteme
Revision as of 15:25, 3 July 2020 by William.mackinnon (talk | contribs)
Overview
EODMS has several options for search and download API access as well as automation i.e. Machine 2 Machine.
As part of the Directive on Open Government, the Government of Canada is committed to improving its #OpenData posture. EODMS API access aims to promote practices per the GC Standards on APIs
To find more GC APIs, visit the GC API site.
https://api.canada.ca/en/homepage
GitHub Repo - Search and Discovery in Python
Checkout out our open source Python module on GitHub which uses the CSW, WCS and REST APIs to automate the ordering of images from EODMS.
REST API
REST Search
SEARCH FIELDS
Default result format is human-readable HTML. Appending &format=json' or '=xml' to the URL behaves as expected.
Each collection has a unique set of search fields here:
https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/collections/NAPL https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/collections/RCMImageProducts https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/collections/Radarsat1
SPATIAL
https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/search?collection=NAPL&query=CATALOG_IMAGE.THE_GEOM_4326 INTERSECTS POLYGON ((-75 45,-75 46,-76 46,-76 45,-75 45))&format=json
TEMPORAL
https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/search?collection=NAPL&query=CATALOG_IMAGE.START_DATETIME>'2000-01-01' AND CATALOG_IMAGE.START_DATETIME<'2005-01-01'
BOOLEAN
https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/search?collection=NAPL&query=CATALOG_IMAGE.PUBLIC_GOOD=TRUE
STRING
https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/search?collection=NAPL&query=ROLL.ROLL_NUMBER='A128' AND PHOTO.PHOTO_NUMBER='0032
REST Order
URL: https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/order HTTP AUTH: Your EODMS username and password HTTP POST XML:
{
"destinations": [],
"items": [
{
"collectionId": "RCMImageProducts",
"recordId": "7189214"
}
]
}
OPENSEARCH API
OPENSEARCH Search
BASIC
https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/opensearch/search?q=*
SPATIAL
https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/opensearch/search?q=*&bbox=-75.71,45.42,-75.69,45.43
OPENSEARCH Order
Using a RESPONSE from the OPENSEARCH Search, you can parse out the collectionId and recordId and submit a corresponding REST Order. e.g. collectionId=Radarsat1 and recordId=1508207
{
"results": [
{
"recordId": "1508207",
"overviewUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/getObject?FeatureID=SERVICE-RSAT1_001-000000000000000000-1508207&ObjectType=Overview&collectionId=Radarsat1",
"collectionId": "Radarsat1",
"metadata2": [
{
"id": "REASON",
"value": "",
"label": "Reason"
},
{
"id": "ARCHIVE_IMAGE.PRODUCT_TYPE",
"value": "SLC",
"label": "Type"
},
{
"id": "RSAT1.SBEAMFULL",
"value": "Fine 2",
"label": "Beam Mode"
},
{
"id": "IDX_SENSOR.SENSOR_NAME",
"value": "RADARSAT-1",
"label": "Satellite"
},
{
"id": "CATALOG_IMAGE.START_DATETIME",
"value": "2007-12-18 22:55:39 UTC",
"label": "Date"
},
{
"id": "RELATED_PRODUCTS",
"value": "{\"headers\":[\"Product Type\",\"LUT_APPLIED\",\"Sampled Pixel Spacing (Panchromatic)\",\"Product Format\",\"Geodetic Terrain Height\"],\"relatedProducts\":[{\"values\":[\"SLC\",\"Unity\",\"4.638299\",\"CEOS\",\"0.0\"]}],\"collectionID\":\"Radarsat1\",\"imageID\":\"1508207\"}",
"label": "Image Info"
},
{
"id": "Metadata",
"value": "dummy value",
"label": "Metadata"
},
{
"id": "CATALOG_IMAGE.STATUS",
"value": "",
"label": "Status"
},
{
"id": "RSAT1.POLARIZATION",
"value": "HH",
"label": "Polarization"
}
],
"rapiOrderUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/order/direct?collection=Radarsat1&recordId=1508207&destination=fill_me_in",
"title": "RS1_M0630930_F2N_20071218_225547_HH_SLC",
"orderExecuteUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/Client/?entryPoint=preview#?cseq=Radarsat1&record=1508207",
"thumbnailUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/getObject?FeatureID=SERVICE-RSAT1_001-000000000000000000-1508207&ObjectType=Thumbview&collectionId=Radarsat1",
"metadataUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/Client/?entryPoint=resultDetails&resultId=1508207&collectionId=Radarsat1",
"isGeorectified": false,
"collectionTitle": "RADARSAT-1 Open Data Products",
"isOrderable": true,
"thisRecordUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/record/Radarsat1/1508207",
"metadata": [
[
"Reason",
""
],
[
"Type",
"SLC"
],
[
"Beam Mode",
"Fine 2"
],
[
"Satellite",
"RADARSAT-1"
],
[
"Date",
"2007-12-18 22:55:39 UTC"
],
[
"Image Info",
"{\"headers\":[\"Product Type\",\"LUT_APPLIED\",\"Sampled Pixel Spacing (Panchromatic)\",\"Product Format\",\"Geodetic Terrain Height\"],\"relatedProducts\":[{\"values\":[\"SLC\",\"Unity\",\"4.638299\",\"CEOS\",\"0.0\"]}],\"collectionID\":\"Radarsat1\",\"imageID\":\"1508207\"}"
],
[
"Metadata",
"dummy value"
],
[
"Status",
""
],
[
"Polarization",
"HH"
]
]
}
...
OPENSEARCH API. Copernicus Hub Inspired. Provides Direct Download. RADARSAT-1 and NAPL, ONLY)
This particular flavour of OpenSearch API is heavily modeled after the [Copernicus Data Hub OpenSearch access program] to align RADARSAT-1 and NAPL access with SENTINEL access. Each API response comes with a direct download URL (in contrast to the REST Order API call which invokes a slower archive-pull-and-stage operation.
QUERY FORMAT
https://data.eodms-sgdot.nrcan-rncan.gc.ca/api/dhus/v1/products/Radarsat1/search?q={searchterms}
QUERY EXAMPLES
https://data.eodms-sgdot.nrcan-rncan.gc.ca/api/dhus/v1/products/Radarsat1/search?q=beginposition:[2007-08-20T00:49:50Z%20TO%202007-08-22T10:50:50Z]
SEARCH TERMS
producttype which can be SCN,SGF, or SLC for Radarsat-1 sensoroperationalmode which refers to the beam mnemonic like EH3, F1, W2, SCWB, SCNA etc.. polarisationmode which is only HH for R1 ingestiondate beginposition endposition footprint (point and polygon)
The search terms can be used together with AND and OR. There is also support for wildcards, free text searching and other operators:
Wildcard Description * any sequence of zero or more characters ? any one character
Operator Description AND Narrow search and retrieve records containing all of the words it separates. OR Broaden search and retrieve records containing any of the words it separates. NOT Narrow search and retrieve records that do not contain the term following it. ( ) Group words or phrases when combining Boolean phrases and to show the order in which relationships should be considered.
Heavily modeled after the ESA Data hubs so the following should apply:
https://colhub.copernicus.eu/userguide/OpenSearchAPI
OGC APIs
The following are usage samples of EODMS' OGC APIs with a full open source Python module on GitHub
Discover some data / CSW GetRecords
URL (POST)
Each data collection has a separate URL. The following are supported:
https://www.eodms-sgdot.nrcan-rncan.gc.ca/MetaManagerCSW/csw/eodms_catalog
https://www.eodms-sgdot.nrcan-rncan.gc.ca/MetaManagerCSW/csw/eodms_napl_catalog
https://www.eodms-sgdot.nrcan-rncan.gc.ca/MetaManagerCSW/csw/RCMImageProducts
HTTP POST - HEADER (No Username/Password)
<csw:GetRecords service='CSW' version='2.0.2' maxRecords='10' startPosition='1' resultType='results' outputFormat='application/xml' outputSchema='http://www.opengis.net/cat/csw/2.0.2' xmlns='http://www.opengis.net/cat/csw/2.0.2' xmlns:csw='http://www.opengis.net/cat/csw/2.0.2' xmlns:ogc='http://www.opengis.net/ogc' xmlns:ows='http://www.opengis.net/ows' xmlns:dc='http://purl.org/dc/elements/1.1/' xmlns:dct='http://purl.org/dc/terms/' xmlns:gml='http://www.opengis.net/gml' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:schemaLocation='http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd'> <csw:Query typeNames='csw:Record'> <csw:ElementSetName typeNames='csw:Record'>summary</csw:ElementSetName> </csw:Query> </csw:GetRecords>
HTTP POST - HEADER (With Username/Password SOAP Envelope)
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header>
<wsse:Security soapenv:mustUnderstand="1" xmlns:wsse="http://docs.oasisopen.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
<wsse:UsernameToken xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/ oasis-200401-wss-wssecurity-utility-1.0.xsd">
<wsse:Username>#####</wsse:Username>
<wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">#####</wsse:Password>
</wsse:UsernameToken>
</wsse:Security>
</soapenv:Header>
<soapenv:Body>
<csw:GetRecords service='CSW' version='2.0.2' maxRecords='10' startPosition='1' resultType='results'
outputFormat='application/xml' outputSchema='http://www.opengis.net/cat/csw/2.0.2'
xmlns='http://www.opengis.net/cat/csw/2.0.2' xmlns:csw='http://www.opengis.net/cat/csw/2.0.2'
xmlns:ogc='http://www.opengis.net/ogc' xmlns:ows='http://www.opengis.net/ows'
xmlns:dc='http://purl.org/dc/elements/1.1/' xmlns:dct='http://purl.org/dc/terms/'
xmlns:gml='http://www.opengis.net/gml' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
xsi:schemaLocation='http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd'>
<csw:Query typeNames='csw:Record'>
<csw:ElementSetName typeNames='csw:Record'>summary</csw:ElementSetName>
</csw:Query>
</csw:GetRecords>
</soapenv:Body>
</soapenv:Envelope>
HTTP RESPONSE
<?xml version="1.0" encoding="UTF-8"?> <GetRecordsResponse xmlns="http://www.opengis.net/cat/csw/2.0.2"> <SearchStatus timestamp="2018-08-28T01:59:38-04:00"/> <SearchResults numberOfRecordsMatched="1949485" numberOfRecordsReturned="10" nextRecord="11"> <csw:SummaryRecord xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:ows="http://www.opengis.net/ows" xmlns:dct="http://purl.org/dc/terms/" xmlns:dc="http://purl.org/dc/elements/1.1/"> <dc:identifier>1511078</dc:identifier> <dc:title>Radarsat-1 Raw Scenes</dc:title> <dc:type>dataset</dc:type> <dc:subject>Satellites, Imaging,Radar, Digital Collection, Mapping, Spatial Data, Remote Sensing </dc:subject> <dc:subject>F1</dc:subject> <dc:format>GeoTIFF</dc:format> <dct:abstract>The Radarsat-1 satellite has a synthetic aperture radar(SAR) imaging instrument. The images are used internationally to manage and monitor the Earth's resources and to monitor global climate change, as well as in many other commercial and scientific applications. RADARSAT-1 is ideally suited to supporting these tasks because of its wide range of beams, SAR technology, frequent revisit period, high-quality products and fast, efficient delivery. Each of Radarsat-1's seven beam modes offer a different image resolution. The modes include Fine, which covers an area of 50 km × 50 km (31 mi × 31 mi) (2,500 km2 (970 sq mi)) with a resolution of 10 metres (33 ft); Standard, which covers an area of 100 km × 100 km (62 mi × 62 mi) (10,000 km2 (3,900 sq mi)) and has a resolution of 30 metres (98 ft); and ScanSAR wide, which covers a 500 km × 500 km (310 mi × 310 mi) (250,000 km2 (97,000 sq mi)) area with a resolution of 100 metres (330 ft). Radarsat-1 also has the unique ability to direct its beam at different angles. </dct:abstract> <ows:BoundingBox> <ows:LowerCorner>32.664853 -3.055419</ows:LowerCorner> <ows:UpperCorner>33.172058 -2.442097</ows:UpperCorner> </ows:BoundingBox> </csw:SummaryRecord> ... </SearchResults> </GetRecordsResponse>
Fetch a Single Record / CSW GetRecordById
URL (GET REQUEST)
HTTP RESPONSE
<GetRecordByIdResponse xmlns="http://www.opengis.net/cat/csw/2.0.2"> <csw:SummaryRecord xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:ows="http://www.opengis.net/ows" xmlns:dct="http://purl.org/dc/terms/" xmlns:dc="http://purl.org/dc/elements/1.1/"> <dc:identifier>1515826</dc:identifier> <dc:title>Radarsat-1 Raw Scenes</dc:title> <dc:type>dataset</dc:type> <dc:subject>Satellites, Imaging,Radar, Digital Collection, Mapping, Spatial Data, Remote Sensing </dc:subject> <dc:subject>F5</dc:subject> <dc:format>GeoTIFF</dc:format> <dct:abstract>The Radarsat-1 satellite has a synthetic aperture radar(SAR) imaging instrument. The images are used internationally to manage and monitor the Earth's resources and to monitor global climate change, as well as in many other commercial and scientific applications. RADARSAT-1 is ideally suited to supporting these tasks because of its wide range of beams, SAR technology, frequent revisit period, high-quality products and fast, efficient delivery. Each of Radarsat-1's seven beam modes offer a different image resolution. The modes include Fine, which covers an area of 50 km × 50 km (31 mi × 31 mi) (2,500 km2 (970 sq mi)) with a resolution of 10 metres (33 ft); Standard, which covers an area of 100 km × 100 km (62 mi × 62 mi) (10,000 km2 (3,900 sq mi)) and has a resolution of 30 metres (98 ft); and ScanSAR wide, which covers a 500 km × 500 km (310 mi × 310 mi) (250,000 km2 (97,000 sq mi)) area with a resolution of 100 metres (330 ft). Radarsat-1 also has the unique ability to direct its beam at different angles. </dct:abstract> <ows:BoundingBox> <ows:LowerCorner>-112.436916 40.41178</ows:LowerCorner> <ows:UpperCorner>-111.769493 40.977687</ows:UpperCorner> </ows:BoundingBox> </csw:SummaryRecord> </GetRecordByIdResponse>
Order that Record / WCS DescribeCoverage
The WCS DescribeCoverage message is used for ordering and can be sent as a POST or GET (recommended) request.
URL (GET REQUEST)
URL (POST)
https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/services/WESOrder/wcs
HTTP POST - HEADER
<wcs:DescribeCoverage service="WCS" version="2.0.1" xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:schemaLocation="http://www.opengis.net/wcs/2.0 http://schemas.opengis.net/wcs/2.0/wcsAll.xsd" xmlns="http://www.opengis.net/wcs/2.0" xmlns:wcs="http://www.opengis.net/wcs/2.0"> <wcs:CoverageId>Radarsat1--5117806</wcs:CoverageId> </wcs:DescribeCoverage>
HTTP RESPONSE
<wcs:CoverageDescriptions xmlns:wcs="http://www.opengis.net/wcs/2.0"> <wcs:CoverageDescription xmlns:gml="http://www.opengis.net/gml/3.2" gml:id="C380b242c-b324-4364-9ea5-7798f2f5e648"> <wcs:CoverageId>Radarsat1--5117806</wcs:CoverageId> <metadata xmlns="http://www.opengis.net/gmlcov/1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/services/ WESSearch/csw/Radarsat1?request=GetRecordById&Id=5117806& version=2.0.2&outputSchema=http://schema.compusult.net/services/ 2.2.0/WESSearch/csw&service=CSW" xlink:title="CSW GetRecordById" xlink:type="simple"/> <gml:domainSet> <gml:Polygon gml:id="C06023ce2-944d-478f-a391-4e6736c53f2e" srsDimension="2" srsName="EPSG:4326" xmlns:gml="http://www.opengis.net/gml/3.2"> <gml:exterior> <gml:LinearRing> <gml:posList> -74.369379 45.510773 -81.488928 46.423416 -82.487757 41.524618 -75.922503 40.612456 -74.369379 45.510773 </gml:posList> </gml:LinearRing> </gml:exterior> </gml:Polygon> </gml:domainSet> <rangeType xmlns="http://www.opengis.net/gmlcov/1.0"/> <wcs:ServiceParameters> <wcs:CoverageSubtype>GridCoverage</wcs:CoverageSubtype> <wcs:nativeFormat/> <wcs:Extension> <DestinationTypes xmlns="http://schema.compusult.net/services/ 2.7.0/WESOrder/wcs"> <Type>Download</Type> </DestinationTypes> <Parameter xmlns="http://schema.compusult.net/services/2.7.0/WESOrder"> <Name>packagingFormat</Name> <Title>Packaging Format</Title> <Description>Packaging Format</Description> <Default>ZIP</Default> <Choices> <Value>ZIP</Value> <Display>Zip</Display> <Description>Zip</Description> <DependentParameters/> </Choices> <Choices> <Value>TAR</Value> <Display>Tar</Display> <Description>Tar</Description> <DependentParameters/> </Choices> <Choices> <Value>TARGZ</Value> <Display>Tar/GZ</Display> <Description>Tar/GZ</Description> <DependentParameters/> </Choices> <Choices> <Value>TAR</Value> <Display>Tar</Display> <Description>Tar</Description> <DependentParameters/> </Choices> <Choices> <Value>TARGZ</Value> <Display>Tar/GZ</Display> <Description>Tar/GZ</Description> <DependentParameters/> </Choices> </Parameter> </wcs:Extension> </wcs:ServiceParameters> </wcs:CoverageDescription> </wcs:CoverageDescriptions>
Github Repo - Web Automation with Katalon
- A Katalon utility to automate website user clicks for searching and ordering.