Difference between revisions of "Deleteme"

From wiki
Jump to navigation Jump to search
Line 61: Line 61:
  
 
=== OPENSEARCH Search ===
 
=== OPENSEARCH Search ===
 +
 +
To see a list of collection names open to GUESTS to supply to the collection query string argument visit: https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/collections
  
 
BASIC
 
BASIC
  
 
<pre style="background-color:#EEFFCC; border-color:#AACC99">
 
<pre style="background-color:#EEFFCC; border-color:#AACC99">
−
https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/opensearch/search?q=*
+
https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/opensearch/search?collection=Radarsat1&q=*
 
</pre>
 
</pre>
  
Line 71: Line 73:
  
 
<pre style="background-color:#EEFFCC; border-color:#AACC99">
 
<pre style="background-color:#EEFFCC; border-color:#AACC99">
−
https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/opensearch/search?q=*&bbox=-75.71,45.42,-75.69,45.43
+
https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/opensearch/search?collection=NAPL&q=*&bbox=-75.71,45.42,-75.69,45.43
 
</pre>
 
</pre>
  

Revision as of 13:17, 18 December 2020

Overview

EODMS has several options for search and download API access as well as automation i.e. Machine 2 Machine. As part of the Directive on Open Government, the Government of Canada is committed to improving its #OpenData posture. EODMS API access aims to promote practices per the GC Standards on APIs. To find more GC APIs, visit the GC API site

GitHub Repo - Search and Download in Python

GitHub Logo.png

Checkout out our open source Python module on GitHub which uses OGC (CSW, WCS) and REST APIs to automate the search and download of images from EODMS.

REST API

REST Search

SEARCH FIELDS

Default result format is human-readable HTML. Appending &format=json' or '=xml' to the URL behaves as expected. Each collection has a unique set of search fields here: 

 https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/collections/NAPL
 https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/collections/RCMImageProducts
 https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/collections/Radarsat1

SPATIAL 

https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/search?collection=NAPL&query=CATALOG_IMAGE.THE_GEOM_4326 INTERSECTS POLYGON ((-75 45,-75 46,-76 46,-76 45,-75 45))&format=json

TEMPORAL 

https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/search?collection=NAPL&query=CATALOG_IMAGE.START_DATETIME>'2000-01-01' AND CATALOG_IMAGE.START_DATETIME<'2005-01-01'

BOOLEAN 

https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/search?collection=NAPL&query=CATALOG_IMAGE.PUBLIC_GOOD=TRUE

STRING 

https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/search?collection=NAPL&query=ROLL.ROLL_NUMBER='A128' AND PHOTO.PHOTO_NUMBER='0032

REST Order

URL: https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/order

HTTP AUTH: Your EODMS username and password

HTTP POST XML: 

{
    "destinations": [], 
    "items": [
        {
            "collectionId": "RCMImageProducts", 
            "recordId": "7189214"
        }
    ]
}

OPENSEARCH API

OPENSEARCH Search

To see a list of collection names open to GUESTS to supply to the collection query string argument visit: https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/collections

BASIC 

https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/opensearch/search?collection=Radarsat1&q=*

SPATIAL 

https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/opensearch/search?collection=NAPL&q=*&bbox=-75.71,45.42,-75.69,45.43

OPENSEARCH Order

Using a RESPONSE from the OPENSEARCH Search, you can parse out the collectionId and recordId and submit a corresponding REST Order. e.g. collectionId=Radarsat1 and recordId=1508207 

{
    "results": [
        {
            "recordId": "1508207",
            "overviewUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/getObject?FeatureID=SERVICE-RSAT1_001-000000000000000000-1508207&ObjectType=Overview&collectionId=Radarsat1",
            "collectionId": "Radarsat1",
            "metadata2": [
                {
                    "id": "REASON",
                    "value": "",
                    "label": "Reason"
                },
                {
                    "id": "ARCHIVE_IMAGE.PRODUCT_TYPE",
                    "value": "SLC",
                    "label": "Type"
                },
                {
                    "id": "RSAT1.SBEAMFULL",
                    "value": "Fine 2",
                    "label": "Beam Mode"
                },
                {
                    "id": "IDX_SENSOR.SENSOR_NAME",
                    "value": "RADARSAT-1",
                    "label": "Satellite"
                },
                {
                    "id": "CATALOG_IMAGE.START_DATETIME",
                    "value": "2007-12-18 22:55:39 UTC",
                    "label": "Date"
                },
                {
                    "id": "RELATED_PRODUCTS",
                    "value": "{\"headers\":[\"Product Type\",\"LUT_APPLIED\",\"Sampled Pixel Spacing (Panchromatic)\",\"Product Format\",\"Geodetic Terrain Height\"],\"relatedProducts\":[{\"values\":[\"SLC\",\"Unity\",\"4.638299\",\"CEOS\",\"0.0\"]}],\"collectionID\":\"Radarsat1\",\"imageID\":\"1508207\"}",
                    "label": "Image Info"
                },
                {
                    "id": "Metadata",
                    "value": "dummy value",
                    "label": "Metadata"
                },
                {
                    "id": "CATALOG_IMAGE.STATUS",
                    "value": "",
                    "label": "Status"
                },
                {
                    "id": "RSAT1.POLARIZATION",
                    "value": "HH",
                    "label": "Polarization"
                }
            ],
            "rapiOrderUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/order/direct?collection=Radarsat1&recordId=1508207&destination=fill_me_in",
            "title": "RS1_M0630930_F2N_20071218_225547_HH_SLC",
            "orderExecuteUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/Client/?entryPoint=preview#?cseq=Radarsat1&record=1508207",
            "thumbnailUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/getObject?FeatureID=SERVICE-RSAT1_001-000000000000000000-1508207&ObjectType=Thumbview&collectionId=Radarsat1",
            "metadataUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/Client/?entryPoint=resultDetails&resultId=1508207&collectionId=Radarsat1",
            "isGeorectified": false,
            "collectionTitle": "RADARSAT-1 Open Data Products",
            "isOrderable": true,
            "thisRecordUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/record/Radarsat1/1508207",
            "metadata": [
                [
                    "Reason",
                    ""
                ],
                [
                    "Type",
                    "SLC"
                ],
                [
                    "Beam Mode",
                    "Fine 2"
                ],
                [
                    "Satellite",
                    "RADARSAT-1"
                ],
                [
                    "Date",
                    "2007-12-18 22:55:39 UTC"
                ],
                [
                    "Image Info",
                    "{\"headers\":[\"Product Type\",\"LUT_APPLIED\",\"Sampled Pixel Spacing (Panchromatic)\",\"Product Format\",\"Geodetic Terrain Height\"],\"relatedProducts\":[{\"values\":[\"SLC\",\"Unity\",\"4.638299\",\"CEOS\",\"0.0\"]}],\"collectionID\":\"Radarsat1\",\"imageID\":\"1508207\"}"
                ],
                [
                    "Metadata",
                    "dummy value"
                ],
                [
                    "Status",
                    ""
                ],
                [
                    "Polarization",
                    "HH"
                ]
            ]
        }
...

OPENSEARCH API. Copernicus Hub Inspired. Provides Direct Download. RADARSAT-1 and NAPL, ONLY)

This particular flavour of OpenSearch API is heavily modeled after the [Copernicus Data Hub OpenSearch access program] to align RADARSAT-1 and NAPL access with SENTINEL access. Each API response comes with a direct download URL (in contrast to the REST Order API call which invokes a slower archive-pull-and-stage operation.

QUERY FORMAT

https://data.eodms-sgdot.nrcan-rncan.gc.ca/api/dhus/v1/products/Radarsat1/search?q={searchterms}

QUERY EXAMPLES

https://data.eodms-sgdot.nrcan-rncan.gc.ca/api/dhus/v1/products/Radarsat1/search?q=footprint:Intersects((-130.496,54.672)) 

https://data.eodms-sgdot.nrcan-rncan.gc.ca/api/dhus/v1/products/Radarsat1/search?q=beginposition:[2007-08-20T00:49:50Z%20TO%202007-08-22T10:50:50Z]

SEARCH TERMS 

producttype which can be SCN,SGF, or SLC for Radarsat-1
sensoroperationalmode which refers to the beam mnemonic like EH3, F1, W2, SCWB, SCNA etc..
polarisationmode which is only HH for R1
ingestiondate
beginposition
endposition
footprint  (point and polygon)

The search terms can be used together with AND and OR. There is also support for wildcards, free text searching and other operators: 

Wildcard  Description

*         any sequence of zero or more characters
?         any one character

Operator  Description

AND       Narrow search and retrieve records containing all of the words it separates.
OR        Broaden search and retrieve records containing any of the words it separates.
NOT       Narrow search and retrieve records that do not contain the term following it.
( )       Group words or phrases when combining Boolean phrases and to show the order in which relationships should be considered.

Heavily modeled after the ESA Data hubs so the following should apply: https://colhub.copernicus.eu/userguide/OpenSearchAPI

OGC APIs

The following are usage samples of EODMS' OGC APIs with a full open source Python module on GitHub

Discover some data / CSW GetRecords

URL (POST)

Each data collection has a separate URL. The following are supported:

https://www.eodms-sgdot.nrcan-rncan.gc.ca/MetaManagerCSW/csw/eodms_catalog

https://www.eodms-sgdot.nrcan-rncan.gc.ca/MetaManagerCSW/csw/eodms_napl_catalog

https://www.eodms-sgdot.nrcan-rncan.gc.ca/MetaManagerCSW/csw/RCMImageProducts

HTTP POST - HEADER (No Username/Password) 

<csw:GetRecords service='CSW' version='2.0.2' maxRecords='10' startPosition='1' resultType='results' 
	outputFormat='application/xml' outputSchema='http://www.opengis.net/cat/csw/2.0.2' 
	xmlns='http://www.opengis.net/cat/csw/2.0.2' xmlns:csw='http://www.opengis.net/cat/csw/2.0.2' 
	xmlns:ogc='http://www.opengis.net/ogc' xmlns:ows='http://www.opengis.net/ows' 
	xmlns:dc='http://purl.org/dc/elements/1.1/' xmlns:dct='http://purl.org/dc/terms/' 
	xmlns:gml='http://www.opengis.net/gml' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' 
	xsi:schemaLocation='http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd'> 
	<csw:Query typeNames='csw:Record'> 
		<csw:ElementSetName typeNames='csw:Record'>summary</csw:ElementSetName> 
	</csw:Query> 
</csw:GetRecords>

HTTP POST - HEADER (With Username/Password SOAP Envelope) 

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
   <soapenv:Header>
      <wsse:Security soapenv:mustUnderstand="1" xmlns:wsse="http://docs.oasisopen.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
         <wsse:UsernameToken xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/ oasis-200401-wss-wssecurity-utility-1.0.xsd">
            <wsse:Username>#####</wsse:Username>
            <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">#####</wsse:Password>
         </wsse:UsernameToken>
     </wsse:Security>
   </soapenv:Header>
   <soapenv:Body>

<csw:GetRecords service='CSW' version='2.0.2' maxRecords='10' startPosition='1' resultType='results' 
	outputFormat='application/xml' outputSchema='http://www.opengis.net/cat/csw/2.0.2' 
	xmlns='http://www.opengis.net/cat/csw/2.0.2' xmlns:csw='http://www.opengis.net/cat/csw/2.0.2' 
	xmlns:ogc='http://www.opengis.net/ogc' xmlns:ows='http://www.opengis.net/ows' 
	xmlns:dc='http://purl.org/dc/elements/1.1/' xmlns:dct='http://purl.org/dc/terms/' 
	xmlns:gml='http://www.opengis.net/gml' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' 
	xsi:schemaLocation='http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd'> 
	<csw:Query typeNames='csw:Record'> 
		<csw:ElementSetName typeNames='csw:Record'>summary</csw:ElementSetName> 
	</csw:Query> 
</csw:GetRecords>

</soapenv:Body>
</soapenv:Envelope>

HTTP RESPONSE 

<?xml version="1.0" encoding="UTF-8"?> 
<GetRecordsResponse xmlns="http://www.opengis.net/cat/csw/2.0.2">
	<SearchStatus timestamp="2018-08-28T01:59:38-04:00"/>
	<SearchResults numberOfRecordsMatched="1949485" numberOfRecordsReturned="10" 
		nextRecord="11">
		<csw:SummaryRecord xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" 
			xmlns:ows="http://www.opengis.net/ows" 
			xmlns:dct="http://purl.org/dc/terms/" 
			xmlns:dc="http://purl.org/dc/elements/1.1/">
			<dc:identifier>1511078</dc:identifier>
			<dc:title>Radarsat-1 Raw Scenes</dc:title>
			<dc:type>dataset</dc:type>
			<dc:subject>Satellites, Imaging,Radar, Digital Collection, Mapping, 
				Spatial Data, Remote Sensing
			</dc:subject>
			<dc:subject>F1</dc:subject>
			<dc:format>GeoTIFF</dc:format>
			<dct:abstract>The Radarsat-1 satellite has a synthetic aperture 
				radar(SAR) imaging instrument. The images are used internationally 
				to manage and monitor the Earth's resources and to monitor global 
				climate change, as well as in many other commercial and scientific 
				applications. RADARSAT-1 is ideally suited to supporting these 
				tasks because of its wide range of beams, SAR technology, frequent 
				revisit period, high-quality products and fast, efficient delivery. 
				Each of Radarsat-1's seven beam modes offer a different image 
				resolution. The modes include Fine, which covers an area of 50 km 
				× 50 km (31 mi × 31 mi) (2,500 km2 (970 sq mi)) with a 
				resolution of 10 metres (33 ft); Standard, which covers an area of 
				100 km × 100 km (62 mi × 62 mi) (10,000 km2 (3,900 sq mi)) and 
				has a resolution of 30 metres (98 ft); and ScanSAR wide, which 
				covers a 500 km × 500 km (310 mi × 310 mi) (250,000 km2 
				(97,000 sq mi)) area with a resolution of 100 metres (330 ft). 
				Radarsat-1 also has the unique ability to direct its beam at 
				different angles.
			</dct:abstract>
			<ows:BoundingBox>
				<ows:LowerCorner>32.664853 -3.055419</ows:LowerCorner>
				<ows:UpperCorner>33.172058 -2.442097</ows:UpperCorner>
			</ows:BoundingBox>
		</csw:SummaryRecord>
		...
	</SearchResults>
</GetRecordsResponse>

Fetch a Single Record / CSW GetRecordById

URL (GET REQUEST)

https://www.eodms-sgdot.nrcan-rncan.gc.ca/MetaManagerCSW/csw/eodms_catalog?request=GetRecordById&id=1515826&service=CSW&version=2.0.2

HTTP RESPONSE 

<GetRecordByIdResponse xmlns="http://www.opengis.net/cat/csw/2.0.2">
	<csw:SummaryRecord xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" 
		xmlns:ows="http://www.opengis.net/ows" 
		xmlns:dct="http://purl.org/dc/terms/" 
		xmlns:dc="http://purl.org/dc/elements/1.1/">
		<dc:identifier>1515826</dc:identifier>
		<dc:title>Radarsat-1 Raw Scenes</dc:title>
		<dc:type>dataset</dc:type>
		<dc:subject>Satellites, Imaging,Radar, Digital Collection, Mapping, 
			Spatial Data, Remote Sensing
		</dc:subject>
		<dc:subject>F5</dc:subject>
		<dc:format>GeoTIFF</dc:format>
		<dct:abstract>The Radarsat-1 satellite has a synthetic aperture radar(SAR) 
			imaging instrument. The images are used internationally to manage and 
			monitor the Earth's resources and to monitor global climate change, 
			as well as in many other commercial and scientific applications. 
			RADARSAT-1 is ideally suited to supporting these tasks because of 
			its wide range of beams, SAR technology, frequent revisit period, 
			high-quality products and fast, efficient delivery. Each of Radarsat-1's 
			seven beam modes offer a different image resolution. The modes include 
			Fine, which covers an area of 50 km × 50 km (31 mi × 31 mi) (2,500 km2 
			(970 sq mi)) with a resolution of 10 metres (33 ft); Standard, which 
			covers an area of 100 km × 100 km (62 mi × 62 mi) (10,000 km2 (3,900 
			sq mi)) and has a resolution of 30 metres (98 ft); and ScanSAR wide, 
			which covers a 500 km × 500 km (310 mi × 310 mi) (250,000 km2 (97,000 
			sq mi)) area with a resolution of 100 metres (330 ft). Radarsat-1 also 
			has the unique ability to direct its beam at different angles.
		</dct:abstract>
		<ows:BoundingBox>
			<ows:LowerCorner>-112.436916 40.41178</ows:LowerCorner>
			<ows:UpperCorner>-111.769493 40.977687</ows:UpperCorner>
		</ows:BoundingBox>
	</csw:SummaryRecord>
</GetRecordByIdResponse>

Order that Record / WCS DescribeCoverage

The WCS DescribeCoverage message is used for ordering and can be sent as a POST or GET (recommended) request.

URL (GET REQUEST)

https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/services/WESOrder/wcs?SERVICE=WCS&version=2.0.1&REQUEST=DescribeCoverage&coverageId=5117806&CollectionId=Radarsat1

URL (POST)

https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/services/WESOrder/wcs

HTTP POST - HEADER 

<wcs:DescribeCoverage service="WCS" version="2.0.1"
	xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
	xsi:schemaLocation="http://www.opengis.net/wcs/2.0
	http://schemas.opengis.net/wcs/2.0/wcsAll.xsd"
	xmlns="http://www.opengis.net/wcs/2.0"
	xmlns:wcs="http://www.opengis.net/wcs/2.0">
	<wcs:CoverageId>Radarsat1--5117806</wcs:CoverageId>
</wcs:DescribeCoverage>

HTTP RESPONSE 

<wcs:CoverageDescriptions xmlns:wcs="http://www.opengis.net/wcs/2.0">
	<wcs:CoverageDescription xmlns:gml="http://www.opengis.net/gml/3.2" 
		gml:id="C380b242c-b324-4364-9ea5-7798f2f5e648">
		<wcs:CoverageId>Radarsat1--5117806</wcs:CoverageId>
		<metadata xmlns="http://www.opengis.net/gmlcov/1.0" 
			xmlns:xlink="http://www.w3.org/1999/xlink" 
			xlink:href="https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/services/
			WESSearch/csw/Radarsat1?request=GetRecordById&Id=5117806&
			version=2.0.2&outputSchema=http://schema.compusult.net/services/
			2.2.0/WESSearch/csw&service=CSW" xlink:title="CSW GetRecordById" 
			xlink:type="simple"/>
		<gml:domainSet>
			<gml:Polygon gml:id="C06023ce2-944d-478f-a391-4e6736c53f2e" 
				srsDimension="2" srsName="EPSG:4326" 
				xmlns:gml="http://www.opengis.net/gml/3.2">
				<gml:exterior>
					<gml:LinearRing>
						<gml:posList>
							-74.369379 45.510773 
							-81.488928 46.423416 
							-82.487757 41.524618 
							-75.922503 40.612456 
							-74.369379 45.510773
						</gml:posList>
					</gml:LinearRing>
				</gml:exterior>
			</gml:Polygon>
		</gml:domainSet>
		<rangeType xmlns="http://www.opengis.net/gmlcov/1.0"/>
		<wcs:ServiceParameters>
			<wcs:CoverageSubtype>GridCoverage</wcs:CoverageSubtype>
			<wcs:nativeFormat/>
			<wcs:Extension>
				<DestinationTypes xmlns="http://schema.compusult.net/services/
					2.7.0/WESOrder/wcs">
					<Type>Download</Type>
				</DestinationTypes>
				<Parameter 
					xmlns="http://schema.compusult.net/services/2.7.0/WESOrder">
					<Name>packagingFormat</Name>
					<Title>Packaging Format</Title>
					<Description>Packaging Format</Description>
					<Default>ZIP</Default>
					<Choices>
						<Value>ZIP</Value>
						<Display>Zip</Display>
						<Description>Zip</Description>
						<DependentParameters/>
					</Choices>
					<Choices>
						<Value>TAR</Value>
						<Display>Tar</Display>
						<Description>Tar</Description>
						<DependentParameters/>
					</Choices>
					<Choices>
						<Value>TARGZ</Value>
						<Display>Tar/GZ</Display>
						<Description>Tar/GZ</Description>
						<DependentParameters/>
					</Choices>
					<Choices>
						<Value>TAR</Value>
						<Display>Tar</Display>
						<Description>Tar</Description>
						<DependentParameters/>
					</Choices>
					<Choices>
						<Value>TARGZ</Value>
						<Display>Tar/GZ</Display>
						<Description>Tar/GZ</Description>
						<DependentParameters/>
					</Choices>
				</Parameter>
			</wcs:Extension>
		</wcs:ServiceParameters>
	</wcs:CoverageDescription>
</wcs:CoverageDescriptions>

Github Repo - Web Automation with Katalon

  • A Katalon utility to automate website user clicks for searching and ordering.
Retrieved from "https://wiki.gccollab.ca/index.php?title=Deleteme&oldid=36344"