Deleteme

From wiki
Revision as of 11:54, 26 November 2019 by Kevin.ballantyne (talk | contribs)
Jump to navigation Jump to search


EODMS is has several options for API Access and automation i.e. Machine 2 Machine. To find more GC APIs, visit the GC API site.

Api-logo.png

https://api.canada.ca/en/homepage

As part of the Directive on Open Government, the Government of Canada is committed to improving its #OpenData posture. EODMS API access aims to promote practices per the GC Standards on APIs

OpenSearch API (for RADARSAT-1 and NAPL)[edit | edit source]

This OpenSearch API is the latest effort to align with giants like ESA and their [Copernicus Data Hub OpenSearch access program] to improve RADARSAT-1 access

To form a get query it would take the form of:

https://data.eodms-sgdot.nrcan-rncan.gc.ca/api/dhus/v1/products/Radarsat1/search?q={searchterms}

e.g.:

https://data.eodms-sgdot.nrcan-rncan.gc.ca/api/dhus/v1/products/Radarsat1/search?q=footprint:Intersects((-130.496,54.672)) 

https://data.eodms-sgdot.nrcan-rncan.gc.ca/api/dhus/v1/products/Radarsat1/search?q=beginposition:[2007-08-20T00:49:50Z%20TO%202007-08-22T10:50:50Z]

SEARCH TERMS 

producttype which can be SCN,SGF, or SLC for Radarsat-1
sensoroperationalmode which refers to the beam mnemonic like EH3, F1, W2, SCWB, SCNA etc..
polarisationmode which is only HH for R1
ingestiondate
beginposition
endposition
footprint  (point and polygon)

The search terms can be used together with AND and OR. There is also support for wildcards, free text searching and other operators: 

Wildcard	Description

*		any sequence of zero or more characters
?		any one character

Operator	Description

AND	        Narrow search and retrieve records containing all of the words it separates.
OR		Broaden search and retrieve records containing any of the words it separates.
NOT		Narrow search and retrieve records that do not contain the term following it.
( )		Group words or phrases when combining Boolean phrases and to show the order in which relationships should be considered.

Heavily modeled after the ESA Data hubs so the following should apply: https://colhub.copernicus.eu/userguide/OpenSearchAPI

OGC APIs[edit | edit source]

See File:EODMS OGC Services.pdf for complete details as well as an end-to-end sample.

CSW GetRecords[edit | edit source]

URL (POST)

https://www.eodms-sgdot.nrcan-rncan.gc.ca/MetaManagerCSW/csw/eodms_catalog

HTTP POST - HEADER 

<csw:GetRecords service='CSW' version='2.0.2' maxRecords='10' startPosition='1' resultType='results' 
	outputFormat='application/xml' outputSchema='http://www.opengis.net/cat/csw/2.0.2' 
	xmlns='http://www.opengis.net/cat/csw/2.0.2' xmlns:csw='http://www.opengis.net/cat/csw/2.0.2' 
	xmlns:ogc='http://www.opengis.net/ogc' xmlns:ows='http://www.opengis.net/ows' 
	xmlns:dc='http://purl.org/dc/elements/1.1/' xmlns:dct='http://purl.org/dc/terms/' 
	xmlns:gml='http://www.opengis.net/gml' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' 
	xsi:schemaLocation='http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd'> 
	<csw:Query typeNames='csw:Record'> 
		<csw:ElementSetName typeNames='csw:Record'>summary</csw:ElementSetName> 
	</csw:Query> 
</csw:GetRecords>

HTTP RESPONSE 

<?xml version="1.0" encoding="UTF-8"?> 
<GetRecordsResponse xmlns="http://www.opengis.net/cat/csw/2.0.2">
	<SearchStatus timestamp="2018-08-28T01:59:38-04:00"/>
	<SearchResults numberOfRecordsMatched="1949485" numberOfRecordsReturned="10" 
		nextRecord="11">
		<csw:SummaryRecord xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" 
			xmlns:ows="http://www.opengis.net/ows" 
			xmlns:dct="http://purl.org/dc/terms/" 
			xmlns:dc="http://purl.org/dc/elements/1.1/">
			<dc:identifier>1511078</dc:identifier>
			<dc:title>Radarsat-1 Raw Scenes</dc:title>
			<dc:type>dataset</dc:type>
			<dc:subject>Satellites, Imaging,Radar, Digital Collection, Mapping, 
				Spatial Data, Remote Sensing
			</dc:subject>
			<dc:subject>F1</dc:subject>
			<dc:format>GeoTIFF</dc:format>
			<dct:abstract>The Radarsat-1 satellite has a synthetic aperture 
				radar(SAR) imaging instrument. The images are used internationally 
				to manage and monitor the Earth's resources and to monitor global 
				climate change, as well as in many other commercial and scientific 
				applications. RADARSAT-1 is ideally suited to supporting these 
				tasks because of its wide range of beams, SAR technology, frequent 
				revisit period, high-quality products and fast, efficient delivery. 
				Each of Radarsat-1's seven beam modes offer a different image 
				resolution. The modes include Fine, which covers an area of 50 km 
				× 50 km (31 mi × 31 mi) (2,500 km2 (970 sq mi)) with a 
				resolution of 10 metres (33 ft); Standard, which covers an area of 
				100 km × 100 km (62 mi × 62 mi) (10,000 km2 (3,900 sq mi)) and 
				has a resolution of 30 metres (98 ft); and ScanSAR wide, which 
				covers a 500 km × 500 km (310 mi × 310 mi) (250,000 km2 
				(97,000 sq mi)) area with a resolution of 100 metres (330 ft). 
				Radarsat-1 also has the unique ability to direct its beam at 
				different angles.
			</dct:abstract>
			<ows:BoundingBox>
				<ows:LowerCorner>32.664853 -3.055419</ows:LowerCorner>
				<ows:UpperCorner>33.172058 -2.442097</ows:UpperCorner>
			</ows:BoundingBox>
		</csw:SummaryRecord>
		...
	</SearchResults>
</GetRecordsResponse>

CSW GetRecordById[edit | edit source]

URL (GET REQUEST)

https://www.eodms-sgdot.nrcan-rncan.gc.ca/MetaManagerCSW/csw/eodms_catalog?request=GetRecordById&id=1515826&service=CSW&version=2.0.2

HTTP RESPONSE 

<GetRecordByIdResponse xmlns="http://www.opengis.net/cat/csw/2.0.2">
	<csw:SummaryRecord xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" 
		xmlns:ows="http://www.opengis.net/ows" 
		xmlns:dct="http://purl.org/dc/terms/" 
		xmlns:dc="http://purl.org/dc/elements/1.1/">
		<dc:identifier>1515826</dc:identifier>
		<dc:title>Radarsat-1 Raw Scenes</dc:title>
		<dc:type>dataset</dc:type>
		<dc:subject>Satellites, Imaging,Radar, Digital Collection, Mapping, 
			Spatial Data, Remote Sensing
		</dc:subject>
		<dc:subject>F5</dc:subject>
		<dc:format>GeoTIFF</dc:format>
		<dct:abstract>The Radarsat-1 satellite has a synthetic aperture radar(SAR) 
			imaging instrument. The images are used internationally to manage and 
			monitor the Earth's resources and to monitor global climate change, 
			as well as in many other commercial and scientific applications. 
			RADARSAT-1 is ideally suited to supporting these tasks because of 
			its wide range of beams, SAR technology, frequent revisit period, 
			high-quality products and fast, efficient delivery. Each of Radarsat-1's 
			seven beam modes offer a different image resolution. The modes include 
			Fine, which covers an area of 50 km × 50 km (31 mi × 31 mi) (2,500 km2 
			(970 sq mi)) with a resolution of 10 metres (33 ft); Standard, which 
			covers an area of 100 km × 100 km (62 mi × 62 mi) (10,000 km2 (3,900 
			sq mi)) and has a resolution of 30 metres (98 ft); and ScanSAR wide, 
			which covers a 500 km × 500 km (310 mi × 310 mi) (250,000 km2 (97,000 
			sq mi)) area with a resolution of 100 metres (330 ft). Radarsat-1 also 
			has the unique ability to direct its beam at different angles.
		</dct:abstract>
		<ows:BoundingBox>
			<ows:LowerCorner>-112.436916 40.41178</ows:LowerCorner>
			<ows:UpperCorner>-111.769493 40.977687</ows:UpperCorner>
		</ows:BoundingBox>
	</csw:SummaryRecord>
</GetRecordByIdResponse>

WCS DescribeCoverage[edit | edit source]

URL (GET REQUEST)

https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/services/WESOrder/wcs?SERVICE=WCS&version=2.0.1&REQUEST=DescribeCoverage&coverageId=5117806&CollectionId=Radarsat1

URL (POST)

https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/services/WESOrder/wcs

HTTP POST - HEADER 

<wcs:DescribeCoverage service="WCS" version="2.0.1"
	xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
	xsi:schemaLocation="http://www.opengis.net/wcs/2.0
	http://schemas.opengis.net/wcs/2.0/wcsAll.xsd"
	xmlns="http://www.opengis.net/wcs/2.0"
	xmlns:wcs="http://www.opengis.net/wcs/2.0">
	<wcs:CoverageId>Radarsat1--5117806</wcs:CoverageId>
</wcs:DescribeCoverage>

HTTP RESPONSE 

<wcs:CoverageDescriptions xmlns:wcs="http://www.opengis.net/wcs/2.0">
	<wcs:CoverageDescription xmlns:gml="http://www.opengis.net/gml/3.2" 
		gml:id="C380b242c-b324-4364-9ea5-7798f2f5e648">
		<wcs:CoverageId>Radarsat1--5117806</wcs:CoverageId>
		<metadata xmlns="http://www.opengis.net/gmlcov/1.0" 
			xmlns:xlink="http://www.w3.org/1999/xlink" 
			xlink:href="https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/services/
			WESSearch/csw/Radarsat1?request=GetRecordById&Id=5117806&
			version=2.0.2&outputSchema=http://schema.compusult.net/services/
			2.2.0/WESSearch/csw&service=CSW" xlink:title="CSW GetRecordById" 
			xlink:type="simple"/>
		<gml:domainSet>
			<gml:Polygon gml:id="C06023ce2-944d-478f-a391-4e6736c53f2e" 
				srsDimension="2" srsName="EPSG:4326" 
				xmlns:gml="http://www.opengis.net/gml/3.2">
				<gml:exterior>
					<gml:LinearRing>
						<gml:posList>
							-74.369379 45.510773 
							-81.488928 46.423416 
							-82.487757 41.524618 
							-75.922503 40.612456 
							-74.369379 45.510773
						</gml:posList>
					</gml:LinearRing>
				</gml:exterior>
			</gml:Polygon>
		</gml:domainSet>
		<rangeType xmlns="http://www.opengis.net/gmlcov/1.0"/>
		<wcs:ServiceParameters>
			<wcs:CoverageSubtype>GridCoverage</wcs:CoverageSubtype>
			<wcs:nativeFormat/>
			<wcs:Extension>
				<DestinationTypes xmlns="http://schema.compusult.net/services/
					2.7.0/WESOrder/wcs">
					<Type>Download</Type>
				</DestinationTypes>
				<Parameter 
					xmlns="http://schema.compusult.net/services/2.7.0/WESOrder">
					<Name>packagingFormat</Name>
					<Title>Packaging Format</Title>
					<Description>Packaging Format</Description>
					<Default>ZIP</Default>
					<Choices>
						<Value>ZIP</Value>
						<Display>Zip</Display>
						<Description>Zip</Description>
						<DependentParameters/>
					</Choices>
					<Choices>
						<Value>TAR</Value>
						<Display>Tar</Display>
						<Description>Tar</Description>
						<DependentParameters/>
					</Choices>
					<Choices>
						<Value>TARGZ</Value>
						<Display>Tar/GZ</Display>
						<Description>Tar/GZ</Description>
						<DependentParameters/>
					</Choices>
					<Choices>
						<Value>TAR</Value>
						<Display>Tar</Display>
						<Description>Tar</Description>
						<DependentParameters/>
					</Choices>
					<Choices>
						<Value>TARGZ</Value>
						<Display>Tar/GZ</Display>
						<Description>Tar/GZ</Description>
						<DependentParameters/>
					</Choices>
				</Parameter>
			</wcs:Extension>
		</wcs:ServiceParameters>
	</wcs:CoverageDescription>
</wcs:CoverageDescriptions>

End-to-End Order Example (using Python)[edit | edit source]

The following steps use Python to search the EODMS_Catalog CSW and WCS services to order a Radarsat-1 image from the EODMS.

For this example, the following restrictions will be place on the query:

  • A bounding box surrounding the City of Ottawa city limits with coordinates:
lower corner: -76.3556 44.9617
upper corner: -75.2466 45.5371
  • All dates before to March 29, 2013 as this was the last day Radarsat-1 transmitted data

Python Packages[edit | edit source]

The only Python package that needs to be installed prior running these scripts is the Requests package (https://realpython.com/python-requests/). It can be installed by running the command pip install requests.

GetRecords POST Request[edit | edit source]

SET VARIABLES

First step is to set the input parameters (restrictions listed above) for the GetRecords request.

Python code: 

# Set specific parameters
lower_corner = '-76.3556 44.9617'
upper_corner = '-75.2466 45.5371'
end_date = '2013-03-29Z'

CREATE REQUEST XML

Next, create the XML POST GetRecords request with the above variables. The request tells the CSW to return the first 15 records based on these variables (or search criteria).

NOTE: To change the number of records returned, change the maxRecords attribute in the XML below to the desired value.

GetRecords POST request example: 

<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords service='CSW' version='2.0.2'
    maxRecords='15'
    startPosition='1'
    resultType='results'
    outputFormat='application/xml'
    outputSchema='http://www.opengis.net/cat/csw/2.0.2'
    xmlns='http://www.opengis.net/cat/csw/2.0.2'
    xmlns:csw='http://www.opengis.net/cat/csw/2.0.2'
    xmlns:ogc='http://www.opengis.net/ogc'
    xmlns:ows='http://www.opengis.net/ows'
    xmlns:dc='http://purl.org/dc/elements/1.1/'
    xmlns:dct='http://purl.org/dc/terms/'
    xmlns:gml='http://www.opengis.net/gml'
    xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
    xsi:schemaLocation='http://www.opengis.net/cat/csw/2.0.2
    http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd'>
    <csw:Query typeNames='csw:Record'>
        <csw:ElementSetName typeNames='csw:Record'>full</csw:ElementSetName>
        <csw:Constraint version="1.1.0">
            <ogc:Filter>
                <ogc:And>
                    <ogc:PropertyIsLessThan>
                        <ogc:PropertyName>dc:date</ogc:PropertyName>
                        <ogc:Literal>%s</ogc:Literal>
                    </ogc:PropertyIsLessThan>
                    <ogc:PropertyIsLike escapeChar='\\' singleChar='?' 
                        wildCard='*'>
                        <ogc:PropertyName>dc:title</ogc:PropertyName>
                        <ogc:Literal>*</ogc:Literal>
                    </ogc:PropertyIsLike>
                    <ogc:BBOX>
                        <ogc:PropertyName>ows:BoundingBox</ogc:PropertyName>
                        <gml:Envelope>
                            <gml:lowerCorner>%s</gml:lowerCorner>
                            <gml:upperCorner>%s</gml:upperCorner>
                        </gml:Envelope>
                    </ogc:BBOX>
                </ogc:And>
         </ogc:Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

Python code of request: 

# Submit a GetRecords to the CSW
post_xml = '''<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords service='CSW' version='2.0.2'
    maxRecords='15'
    startPosition='1'
    resultType='results'
    outputFormat='application/xml'
    outputSchema='http://www.opengis.net/cat/csw/2.0.2'
    xmlns='http://www.opengis.net/cat/csw/2.0.2'
    xmlns:csw='http://www.opengis.net/cat/csw/2.0.2'
    xmlns:ogc='http://www.opengis.net/ogc'
    xmlns:ows='http://www.opengis.net/ows'
    xmlns:dc='http://purl.org/dc/elements/1.1/'
    xmlns:dct='http://purl.org/dc/terms/'
    xmlns:gml='http://www.opengis.net/gml'
    xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
    xsi:schemaLocation='http://www.opengis.net/cat/csw/2.0.2
    http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd'>
    <csw:Query typeNames='csw:Record'>
        <csw:ElementSetName typeNames='csw:Record'>full</csw:ElementSetName>
        <csw:Constraint version="1.1.0">
            <ogc:Filter>
                <ogc:And>
                    <ogc:PropertyIsLessThan>
                        <ogc:PropertyName>dc:date</ogc:PropertyName>
                        <ogc:Literal>%s</ogc:Literal>
                    </ogc:PropertyIsLessThan>
                    <ogc:PropertyIsLike escapeChar='\\' singleChar='?' 
                        wildCard='*'>
                        <ogc:PropertyName>dc:title</ogc:PropertyName>
                        <ogc:Literal>*</ogc:Literal>
                    </ogc:PropertyIsLike>
                    <ogc:BBOX>
                        <ogc:PropertyName>ows:BoundingBox</ogc:PropertyName>
                        <gml:Envelope>
                            <gml:lowerCorner>%s</gml:lowerCorner>
                            <gml:upperCorner>%s</gml:upperCorner>
                        </gml:Envelope>
                    </ogc:BBOX>
                </ogc:And>
         </ogc:Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>''' % (end_date, lower_corner, upper_corner)

SEND THE REQUEST

The next step is to send the request to the WCS URL. In Python, use the requests object to send a POST request: 

csw_url = 'https://www.eodms-sgdot.nrcan-rncan.gc.ca/MetaManagerCSW' \
            '/csw/eodms_catalog'
headers = {'Content-Type':'application/xml'}
csw_r = requests.post(csw_url, data=post_xml)

The GetRecords response will contain all the records up to the maxRecords value (in this case the first 15). The following HTTP response will be returned from the CSW: 

<?xml version="1.0" encoding="UTF-8"?>
<GetRecordsResponse xmlns="http://www.opengis.net/cat/csw/2.0.2">
    <SearchStatus timestamp="2019-11-22T04:11:02-05:00"/>
    <SearchResults numberOfRecordsMatched="705" 
        numberOfRecordsReturned="15" nextRecord="16">
        <csw:Record xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" 
            xmlns:ows="http://www.opengis.net/ows" 
            xmlns:dct="http://purl.org/dc/terms/" 
            xmlns:dc="http://purl.org/dc/elements/1.1/">
            <dc:identifier>1508208</dc:identifier>
            <dc:title>Radarsat-1 Raw Scenes</dc:title>
            <dc:type>dataset</dc:type>
            <dc:subject>Satellites, Imaging,Radar, Digital Collection, 
                Mapping, Spatial Data, Remote Sensing
            </dc:subject>
            <dc:subject>F2</dc:subject>
            <dc:format>GeoTIFF</dc:format>
            <dc:creator>Government of Canada; Natural Resources Canada; 
                Earth Sciences Sector; Canada Centre for Mapping and 
                Earth Observation
            </dc:creator>
            <dc:description>The Radarsat-1 satellite has a synthetic 
                aperture radar(SAR) imaging instrument. The images are 
                used internationally to manage and monitor the Earth's 
                resources and to monitor global climate change, as well 
                as in many other commercial and scientific applications. 
                RADARSAT-1 is ideally suited to supporting these tasks 
                because of its wide range of beams, SAR technology, 
                frequent revisit period, high-quality products and fast, 
                efficient delivery. Each of Radarsat-1's seven beam 
                modes offer a different image resolution. The modes 
                include Fine, which covers an area of 50 km × 50 km 
                (31 mi × 31 mi) (2,500 km2 (970 sq mi)) with a resolution 
                of 10 metres (33 ft); Standard, which covers an area of 
                100 km × 100 km (62 mi × 62 mi) (10,000 km2 (3,900 sq mi)) 
                and has a resolution of 30 metres (98 ft); and ScanSAR 
                wide, which covers a 500 km × 500 km (310 mi × 310 mi) 
                (250,000 km2 (97,000 sq mi)) area with a resolution of 
                100 metres (330 ft). Radarsat-1 also has the unique 
                ability to direct its beam at different angles.
            </dc:description>
            <dct:abstract>The Radarsat-1 satellite has a synthetic 
                aperture radar(SAR) imaging instrument. The images are 
                used internationally to manage and monitor the Earth's 
                resources and to monitor global climate change, as well 
                as in many other commercial and scientific applications. 
                RADARSAT-1 is ideally suited to supporting these tasks 
                because of its wide range of beams, SAR technology, 
                frequent revisit period, high-quality products and fast, 
                efficient delivery. Each of Radarsat-1's seven beam 
                modes offer a different image resolution. The modes 
                include Fine, which covers an area of 50 km × 50 km 
                (31 mi × 31 mi) (2,500 km2 (970 sq mi)) with a resolution 
                of 10 metres (33 ft); Standard, which covers an area of 
                100 km × 100 km (62 mi × 62 mi) (10,000 km2 (3,900 sq mi)) 
                and has a resolution of 30 metres (98 ft); and ScanSAR 
                wide, which covers a 500 km × 500 km (310 mi × 310 mi) 
                (250,000 km2 (97,000 sq mi)) area with a resolution of 
                100 metres (330 ft). Radarsat-1 also has the unique 
                ability to direct its beam at different angles.
            </dct:abstract>
            <dc:publisher>NRCan/CCMEO/EODMS section head</dc:publisher>
            <dc:date>2013-02-21</dc:date>
            <dc:language>eng; CAN</dc:language>
            <dct:references scheme="OVERVIEW">
                https://was-eodms.compusult.net/wes/getObject?
                FeatureID=SERVICE-RSAT1_001-000000000000000000-1508208
                &ObjectType=Thumbview&collectionId=Radarsat1
            </dct:references>
            <dct:references scheme="THUMBVIEW">
                https://was-eodms.compusult.net/wes/getObject?
                FeatureID=SERVICE-RSAT1_001-000000000000000000-1508208
                &ObjectType=Thumbview&collectionId=Radarsat1
            </dct:references>
            <dct:references scheme="transferOptions">
                http://gs.mdacorporation.com/SatelliteData/
                Radarsat1/Radarsat1.aspx
            </dct:references>
            <dct:references scheme="transferOptions">
                http://ceocat.ccrs.nrcan.gc.ca
            </dct:references>
            <dc:source>
                https://www.eodms-sgdot.nrcan-rncan.gc.ca:80/MetaManagerCSW
                /csw/eodms_catalog?service=CSW&request=GetRecordById
                &version=2.0.2&ElementSetName=full&outputSchema=http://
                www.isotc211.org/2005/gmd&Id=1508208&outputFormat=text/html
            </dc:source>
            <ows:BoundingBox>
                <ows:LowerCorner>-76.275153 45.191208</ows:LowerCorner>
                <ows:UpperCorner>-75.444661 45.739692</ows:UpperCorner>
            </ows:BoundingBox>
        </csw:Record>
        <csw:Record xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" 
            xmlns:ows="http://www.opengis.net/ows" 
            xmlns:dct="http://purl.org/dc/terms/" 
            xmlns:dc="http://purl.org/dc/elements/1.1/">
            .
            .
            .
        </csw:Record>
        .
        .
        .
    </SearchResults>
</GetRecordsResponse>

EXTRACT RECORD ID AND COLLECTION ID FROM RESPONSE

Using the GetRecords HTTP response, the record ID and the collection ID can be extracted. The record ID is taken from <dc:identifier> in the response. The collection ID can be extracted by parsing the URL in the <dct:references>.

The next step in the script is to convert the XML response into an ElementTree and get the XML element of the first record (or remove the break and add each rec_element to a list to go through each record in the response): 

# Get first record from the response XML
resp_xml = csw_r.content
root = ElementTree.fromstring(resp_xml)

record_tag = '{http://www.opengis.net/cat/csw/2.0.2}Record'
for child in root.iter('*'):
    if child.find(record_tag):
        rec_element = child.find(record_tag)
        break

Using the record element, locate the <dc:identifier> element and get its text: 

# Get the ID of the first record
id_tag = '{http://purl.org/dc/elements/1.1/}identifier'
id_el = rec_element.find(id_tag)
rec_id = id_el.text

Next, locate the <dct:references> in the record element and parse the URL from it: 

# Parse the collection ID from the first dct:references links
ref_tag = '{http://purl.org/dc/terms/}references'
ref_el = rec_element.find(ref_tag)
ref_url = ref_el.text

url_parse = urlparse(ref_url)
url_query = url_parse.query
query_items = {u.split('=')[0]:u.split('=')[1] for u in url_query.split('&')}
collection_id = query_items['collectionId']

DescribeCoverage Request[edit | edit source]

The DescribeCoverage feature can be sent as a POST or GET request.

The URL for the DescribeCoverage GET request is:

https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/services/WESOrder/wcs?SERVICE=WCS&version=2.0.1&REQUEST=DescribeCoverage&coverageId=1508208&CollectionId=Radarsat1

The POST request would be: 

<?xml version="1.0" encoding="UTF-8"?>
<wcs:DescribeCoverage service="WCS" version="2.0.1"
    xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
    xsi:schemaLocation="http://www.opengis.net/wcs/2.0
    http://schemas.opengis.net/wcs/2.0/wcsAll.xsd"
    xmlns="http://www.opengis.net/wcs/2.0"
    xmlns:wcs="http://www.opengis.net/wcs/2.0">
    <wcs:CoverageId>Radarsat1--5117806</wcs:CoverageId>
</wcs:DescribeCoverage>

For the Python scripts, the GET URL will be used. The code for the DescribeCoverage GET URL is: 

# Submit a DescribeCoverage GET request to the WCS using the record and collection IDs
wcs_url = 'https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/services/WESOrder/' \
    'wcs?SERVICE=WCS&version=2.0.1&REQUEST=DescribeCoverage' \
    '&coverageId=%s&CollectionId=%s' % (rec_id, collection_id)

Using the WCS service requires a user account. To send an authenticated request in Python, a session containing the username and password has to be created first. The following code creates a session with username and password and then sends the GET request through the session: 

username = '######'
password = base64.b64decode("######").decode("utf-8")

session = requests.Session()
session.auth = (username, password)
wcs_desccov = session.get(url=wcs_url)

Next, the DescribeCoverage will send back the following response if the request was successful: 

<wcs:CoverageDescriptions xmlns:wcs="http://www.opengis.net/wcs/2.0">
    <wcs:CoverageDescription xmlns:gml="http://www.opengis.net/gml/3.2" 
        gml:id="Cd5a023dc-0003-4ddf-b5cc-5d2e533eac05">
        <wcs:CoverageId>Radarsat1--1508208</wcs:CoverageId>
        <metadata xmlns="http://www.opengis.net/gmlcov/1.0" 
            xmlns:xlink="http://www.w3.org/1999/xlink" 
            xlink:href="https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes
            /services/WESSearch/csw/Radarsat1?request=GetRecordById
            &Id=1508208&version=2.0.2&outputSchema=http://schema.compusult.net/
            services/2.2.0/WESSearch/csw&service=CSW" 
            xlink:title="CSW GetRecordById" 
            xlink:type="simple"/>
        <gml:domainSet>
            <gml:Polygon xmlns:gml="http://www.opengis.net/gml/3.2" 
            gml:id="C49b46f31-cbe9-433b-9201-d4f0cc04ee0d" 
            srsDimension="2" srsName="EPSG:4326">
                <gml:exterior>
                    <gml:LinearRing>
                        <gml:posList>
                            -76.275153 45.652944 
                            -76.157606 45.191208 
                            -75.444661 45.278003 
                            -75.556206 45.739692 
                            -76.275153 45.652944
                        </gml:posList>
                    </gml:LinearRing>
                </gml:exterior>
            </gml:Polygon>
        </gml:domainSet>
        <rangeType xmlns="http://www.opengis.net/gmlcov/1.0"/>
        <wcs:ServiceParameters>
            <wcs:CoverageSubtype>GridCoverage</wcs:CoverageSubtype>
            <wcs:nativeFormat/>
            <wcs:Extension>
                <DestinationTypes xmlns="http://schema.compusult.net/
                    services/2.7.0/WESOrder/wcs">
                    <Type>Download</Type>
                </DestinationTypes>
                <Parameter xmlns="http://schema.compusult.net/services/
                    2.7.0/WESOrder">
                    <Name>packagingFormat</Name>
                    <Title>Packaging Format</Title>
                    <Description>Packaging Format</Description>
                    <Default>ZIP</Default>
                    <Choices>
                        <Value>ZIP</Value>
                        <Display>Zip</Display>
                        <Description>Zip</Description>
                        <DependentParameters/>
                    </Choices>
                    <Choices>
                        <Value>TAR</Value>
                        <Display>Tar</Display>
                        <Description>Tar</Description>
                        <DependentParameters/>
                    </Choices>
                    <Choices>
                        <Value>TARGZ</Value>
                        <Display>Tar/GZ</Display>
                        <Description>Tar/GZ</Description>
                        <DependentParameters/>
                    </Choices>
                    <Choices>
                        <Value>TAR</Value>
                        <Display>Tar</Display>
                        <Description>Tar</Description>
                        <DependentParameters/>
                    </Choices>
                    <Choices>
                        <Value>TARGZ</Value>
                        <Display>Tar/GZ</Display>
                        <Description>Tar/GZ</Description>
                        <DependentParameters/>
                    </Choices>
                </Parameter>
            </wcs:Extension>
        </wcs:ServiceParameters>
    </wcs:CoverageDescription>
</wcs:CoverageDescriptions>

GetCoverage Request[edit | edit source]

If the request was successful, the status code of 200 will be returned with the request and the GetCoverage operation can be sent to the WCS. A GET request similar to the DescribeCoverage can be sent to the WCS (adding format=applicationgml+xml to the URL query).

In this case, a POST request is used. For the POST request, the CoverageId must be in the format of <collection>--<record_id> so in this case the CoverageId is Radarsat1--1508208: 

if wcs_desccov.status_code == 200:
    
    cov_id = '%s--%s' % (collection_id, rec_id)

    # Submit a GetCoverage to the WCS
    getcov_post = '''<wcs:GetCoverage
    xmlns:wos="http://schema.compusult.net/services/2.6.1/WESOrder"
    xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
    xsi:schemaLocation="http://www.opengis.net/wcs/2.0
    http://schemas.opengis.net/wcs/2.0/wcsAll.xsd"
    xmlns="http://www.opengis.net/wcs/2.0"
    xmlns:wcs="http://www.opengis.net/wcs/2.0"
    xmlns:wes="http://schema.compusult.net/services/2.6.1/WESOrder/wcs"
    service="WCS" version="2.0.1">
    <wcs:CoverageId>%s</wcs:CoverageId>
    <wcs:format>application/gml+xml</wcs:format>
</wcs:GetCoverage>''' % cov_id

wcs_getcov = session.post(url=wcs_url, data=getcov_post)

Once the request has been sent, the user will receive an “EODMS Image Request Submitted Notification” email letting them know that their request has been submitted. Shortly after receiving this email, the user should then receive another email called “EODMS Image Request Delivery Notification” with download links for the specific image.

Full Script[edit | edit source]

import sys
import requests
from xml.etree import ElementTree
import base64
from urllib.parse import urlencode
from urllib.parse import urlparse

def main():

    # Set specific parameters
    aoi = '-76.3556 44.9617 -75.2466 44.9617 -75.2466 45.5371 ' \
            '-76.3556 45.5371 -76.3556 44.9617'
    lower_corner = '-76.3556 44.9617'
    upper_corner = '-75.2466 45.5371'
    end_date = '2013-03-29Z'

    # Submit a GetRecords to the CSW
    post_xml = '''<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords service='CSW' version='2.0.2'
    maxRecords='15'
    startPosition='1'
    resultType='results'
    outputFormat='application/xml'
    outputSchema='http://www.opengis.net/cat/csw/2.0.2'
    xmlns='http://www.opengis.net/cat/csw/2.0.2'
    xmlns:csw='http://www.opengis.net/cat/csw/2.0.2'
    xmlns:ogc='http://www.opengis.net/ogc'
    xmlns:ows='http://www.opengis.net/ows'
    xmlns:dc='http://purl.org/dc/elements/1.1/'
    xmlns:dct='http://purl.org/dc/terms/'
    xmlns:gml='http://www.opengis.net/gml'
    xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
    xsi:schemaLocation='http://www.opengis.net/cat/csw/2.0.2
    http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd'>
    <csw:Query typeNames='csw:Record'>
        <csw:ElementSetName typeNames='csw:Record'>full</csw:ElementSetName>
        <csw:Constraint version="1.1.0">
            <ogc:Filter>
                <ogc:And>
                    <ogc:PropertyIsLessThan>
                        <ogc:PropertyName>dc:date</ogc:PropertyName>
                        <ogc:Literal>%s</ogc:Literal>
                    </ogc:PropertyIsLessThan>
                    <ogc:PropertyIsLike escapeChar='\\' singleChar='?' wildCard='*'>
                        <ogc:PropertyName>dc:title</ogc:PropertyName>
                        <ogc:Literal>*</ogc:Literal>
                    </ogc:PropertyIsLike>
                    <ogc:BBOX>
                        <ogc:PropertyName>ows:BoundingBox</ogc:PropertyName>
                        <gml:Envelope>
                            <gml:lowerCorner>%s</gml:lowerCorner>
                            <gml:upperCorner>%s</gml:upperCorner>
                        </gml:Envelope>
                    </ogc:BBOX>
                </ogc:And>
         </ogc:Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>''' % (end_date, lower_corner, upper_corner)

    csw_url = 'https://www.eodms-sgdot.nrcan-rncan.gc.ca/MetaManagerCSW' \
                '/csw/eodms_catalog'
    headers = {'Content-Type':'application/xml'}
    csw_r = requests.post(csw_url, data=post_xml)
        
    # Get first record from the response XML
    resp_xml = csw_r.content
    root = ElementTree.fromstring(resp_xml)
    
    record_tag = '{http://www.opengis.net/cat/csw/2.0.2}Record'
    for child in root.iter('*'):
        if child.find(record_tag):
            rec_element = child.find(record_tag)
            break
    
    # Get the ID of the first record
    id_tag = '{http://purl.org/dc/elements/1.1/}identifier'
    id_el = rec_element.find(id_tag)
    rec_id = id_el.text
    
    # Parse the collection ID from the first dct:references links
    ref_tag = '{http://purl.org/dc/terms/}references'
    ref_el = rec_element.find(ref_tag)
    ref_url = ref_el.text
    
    url_parse = urlparse(ref_url)
    url_query = url_parse.query
    query_items = {u.split('=')[0]:u.split('=')[1] \
                  for u in url_query.split('&')}
    collection_id = query_items['collectionId']
    
    # Submit a DescribeCoverage GET request to the WCS using the record 
    #   and collection IDs
    wcs_url = 'https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/services/WESOrder/' \
            'wcs?SERVICE=WCS&version=2.0.1&REQUEST=DescribeCoverage' \
            '&coverageId=%s&CollectionId=%s' % (rec_id, collection_id)
    
    # Create a session with authentication
    username = 'keballan'
    password = base64.b64decode("TWJWbUNoMTIj").decode("utf-8")
    
    session = requests.Session()
    session.auth = (username, password)
    wcs_desccov = session.get(url=wcs_url)
    
    if wcs_desccov.status_code == 200:
    
        cov_id = '%s--%s' % (collection_id, rec_id)
    
        # Submit a GetCoverage to the WCS
        getcov_post = '''<wcs:GetCoverage
    xmlns:wos="http://schema.compusult.net/services/2.6.1/WESOrder"
    xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
    xsi:schemaLocation="http://www.opengis.net/wcs/2.0
    http://schemas.opengis.net/wcs/2.0/wcsAll.xsd"
    xmlns="http://www.opengis.net/wcs/2.0"
    xmlns:wcs="http://www.opengis.net/wcs/2.0"
    xmlns:wes="http://schema.compusult.net/services/2.6.1/WESOrder/wcs"
    service="WCS" version="2.0.1">
    <wcs:CoverageId>%s</wcs:CoverageId>
    <wcs:format>application/gml+xml</wcs:format>
</wcs:GetCoverage>''' % cov_id

        wcs_getcov = session.post(url=wcs_url, data=getcov_post)

if __name__ == '__main__':
    sys.exit(main())

Katalon[edit | edit source]

  • A Katalon utility to automate website user clicks for searching and ordering.


Retrieved from "https://wiki.gccollab.ca/index.php?title=Deleteme&oldid=14248"