Difference between revisions of "Data Catalog"

From wiki
Jump to navigation Jump to search
 
(23 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
== Data Catalog ==
+
== Data Catalogue ==
−
Data catalog is an organized service that allows users to centralize metadata and learn more about their data sources which help organizations achieve more values from their assets.
+
Data Catalogue is an organized service that allows users to centralize [[Metadata | Metadata]] and learn more about their data sources which help organizations achieve more values from their assets.
−
Following are some advantages to centralizing metadata:<br>
+
Following are some advantages to centralizing [[Metadata | Metadata]]:<br>
  
 
* Consistency and accuracy across the department/government
 
* Consistency and accuracy across the department/government
Line 8: Line 8:
 
* Allow users to self-serve
 
* Allow users to self-serve
  
−
== Why Data Catalog ==
+
== Why Data Catalogue ==
 
[[File:USGS DataLifeCycle.png|thumb|188x188px|
 
[[File:USGS DataLifeCycle.png|thumb|188x188px|
 
USGS data Life Cycle
 
USGS data Life Cycle
Line 15: Line 15:
 
Data-driven culture empowers users with getting access to their data source. With this, the growing numbers of cloud applications, privacy regulations and security rules are making it more difficult to effectively secure and govern data<br>
 
Data-driven culture empowers users with getting access to their data source. With this, the growing numbers of cloud applications, privacy regulations and security rules are making it more difficult to effectively secure and govern data<br>
  
−
Therefor, a Data Catalog is needed to:
+
Therefore, a Data Catalogue is needed to:
  
 
* Spend less time searching for data and more time using it to gain insight
 
* Spend less time searching for data and more time using it to gain insight
 
* Have a better/safer access to data through governance. Departments have a hard time finding data and controlling who has access to it.
 
* Have a better/safer access to data through governance. Departments have a hard time finding data and controlling who has access to it.
 
* Reduce the cost of data redundancy and hoarding.
 
* Reduce the cost of data redundancy and hoarding.
−
* Better linkage between the technical value of metadata and the business value.
+
* Better linkage between the technical value of [[Metadata | Metadata]] and the business value.
 +
 
 +
== Data Catalogue Architecture ==
 +
'''A work in progress Architecture'''
 +
 
 +
As explained in the following Diagram, the Catalogue will be the entry point for [[TADAP| TADAP]]
 +
 
 +
When a user requires to add a data set to [[TADAP| TADAP]], the Catalogue will be able to process this request and help the user create the metadata that goes with it, data is uploaded.
 +
 
 +
1- How the Catalogue will be connected to [[TADAP| TADAP]]
 +
 
 +
[[File:Catalogue_TADAP.png|545x545px]]
 +
 
 +
 +
2- How data is published to FGP and Open Data Portal
 +
{| class="wikitable"
 +
|-
 +
! Catalogue Conceptual Architecure !! How data is published to FGP and Open Data Portal
 +
|-
 +
|  [[File:ConceptualArchitecture2.png|545x545px]] || [[File:Catalog_FGP_OpenData_Publish.png|545x545px]]
 +
|}
  
−
== Data Catalog Architecture ==
 
−
<gallery>
 
−
File:CatalogArchitecture.png|A work in progress DFO-MPO Data Catalog Conceptual Architecture
 
−
</gallery>
 
 
== List of Stakeholders ==
 
== List of Stakeholders ==
−
'''Please note this is an initial list of stakeholders to collect their requirements'''
+
'''Please note this is an initial list.'''
 +
 
 +
'''Please feel free to contact us if you think there's room for improvements.'''  
 
{| class="wikitable mw-collapsible"
 
{| class="wikitable mw-collapsible"
 
|+
 
|+
Line 70: Line 88:
 
|Theraesa Coyle and Johannie Duhaime
 
|Theraesa Coyle and Johannie Duhaime
 
|Aquaculture Monitoring Program
 
|Aquaculture Monitoring Program
 +
|-
 +
|Ocean Science Division
 +
|Di Wan
 +
|Scientific oceanographic survey data management
 
|-
 
|-
 
|Fishery & Assessment Data  Section, Science Branch
 
|Fishery & Assessment Data  Section, Science Branch
−
|Bruce A. Patten and Di Wan
+
|Bruce A. Patten
 
|Scientific survey documentation for fish population  assessment, fish harvest tracking and reporting
 
|Scientific survey documentation for fish population  assessment, fish harvest tracking and reporting
 
|-
 
|-
Line 88: Line 110:
 
|-
 
|-
 
|Information Management and  Policy Strategies
 
|Information Management and  Policy Strategies
−
|Lyn Warner
+
|Annette Anthony/Sylvie Boucher
 
|Open Data Portal
 
|Open Data Portal
 
|-
 
|-
Line 96: Line 118:
 
|}
 
|}
  
−
== Data Catalog Team ==
+
== Data Catalogue Team ==
 +
please feel free to contact us to discuss the project and how you can participate
 +
 
 +
{|
 +
|'''Name'''
 +
!
 +
|-
 +
|George Esper
 +
|
 +
|-
 +
|Riham Elhabyan
 +
|
 +
|-
 +
|David Cornwell
 +
|
 +
|-
 +
|Yask Shelat
 +
|
 +
|-
 +
|Abdul K Hamdo
 +
|
 +
|}

Latest revision as of 10:01, 19 March 2020

Data Catalogue

Data Catalogue is an organized service that allows users to centralize Metadata and learn more about their data sources which help organizations achieve more values from their assets. Following are some advantages to centralizing Metadata:

  • Consistency and accuracy across the department/government
  • Better data congruency, quality and structure
  • Makes data easily accessible
  • Allow users to self-serve

Why Data Catalogue

USGS data Life Cycle

In an enterprise vision, we would like to minimize the number of data silos, get a faster access to what matters most, and function as a single source (during the Data Life Cycle) for better doing.
Data-driven culture empowers users with getting access to their data source. With this, the growing numbers of cloud applications, privacy regulations and security rules are making it more difficult to effectively secure and govern data

Therefore, a Data Catalogue is needed to:

  • Spend less time searching for data and more time using it to gain insight
  • Have a better/safer access to data through governance. Departments have a hard time finding data and controlling who has access to it.
  • Reduce the cost of data redundancy and hoarding.
  • Better linkage between the technical value of Metadata and the business value.

Data Catalogue Architecture

A work in progress Architecture

As explained in the following Diagram, the Catalogue will be the entry point for TADAP

When a user requires to add a data set to TADAP, the Catalogue will be able to process this request and help the user create the metadata that goes with it, data is uploaded.

1- How the Catalogue will be connected to TADAP

545x545px


2- How data is published to FGP and Open Data Portal

Catalogue Conceptual Architecure How data is published to FGP and Open Data Portal
ConceptualArchitecture2.png Catalog FGP OpenData Publish.png

List of Stakeholders

Please note this is an initial list.

Please feel free to contact us if you think there's room for improvements.

Sector Contact Person Project / Program
Conservation and Protection Lise Melanson Department Violation System (DVS)
Pauline Lalonde C-STAT
Canada Coast Guard Shawn Legault and Nicholas O’Hara SIPA
Jean-François Coutu INNAV
Chris Burnie-Gardiner and Patrick Marion SISAR
Bert Paulin Reporting for CCG programs
Canadian Hydrographic Services Terry Fanning and Matthew McGowan Main work is to move science, ocean, and species data to the Government of Canada Open Portal
Claude Guay Metadata management and data management for whole science sector
SRS, Science Branch Theraesa Coyle and Johannie Duhaime Aquaculture Monitoring Program
Ocean Science Division Di Wan Scientific oceanographic survey data management
Fishery & Assessment Data Section, Science Branch Bruce A. Patten Scientific survey documentation for fish population assessment, fish harvest tracking and reporting
Aquaculture Management Directorate, Aquatic Ecosystems Sector Tyree Lush and Arsenault Shane Canadian Shellfish Sanitation Program (CSSP) mapping system
Fisheries & Licence Policy / Fisheries and Harbour Management Mark Ledwell FHM
Ecosystems and Fisheries Management: System Integration Aaron Gillis and Andrew Frost EFM-SI
Information Management and Policy Strategies Annette Anthony/Sylvie Boucher Open Data Portal
Ocean Data and Information Section Tobias Spears To be determined

Data Catalogue Team

please feel free to contact us to discuss the project and how you can participate

Name
George Esper
Riham Elhabyan
David Cornwell
Yask Shelat
Abdul K Hamdo
Retrieved from "https://wiki.gccollab.ca/index.php?title=Data_Catalog&oldid=19275"