AI initiative registry

Every initiative listed in the registry can be consulted here, without an account. Each record shows its progress, its milestones and the history of its decisions. Requests still under triage do not appear: they join the registry once reviewed.

Perception and Understanding

AI that interprets the world, whether through text, images, or sound.

What is “Perception and Understanding”?

This category includes AI systems that sense, interpret, or perceive data from the environment, emulating human senses like sight, sound, touch, smell, and language comprehension. These capabilities form the foundation of AI awareness, enabling systems to extract meaning from unstructured inputs like text, images, video, audio, or environmental signals.

Perception and Understanding
Subcategory	Description	Examples
Language understanding — Natural Language Processing (NLP)	Understanding and generating human language	Chatbots, document summarization
Computer vision	Understanding images, video, and spatial data	Object detection, satellite imagery analysis, species ID
Speech recognition	Converting audio to text	Voice-to-text for transcription, voice-based commands for virtual assistants
Multimodal AI	Combining input types (text + images, etc.)	Image captioning, document understanding tools
Olfactory AI (artificial smell recognition)	Detecting and interpreting smells using e-noses	Gas leak detection, spoilage sensing, pollutant ID in fish labs
Haptic AI (artificial touch sensing)	Interpreting touch and tactile feedback	Robotic sampling, pressure detection, texture, temperature
Audio event detection	Recognizing non-speech sounds	Marine mammal calls (whale song), mechanical alerts, engine noise anomaly detection
Sensor fusion AI	Combining multiple sensory inputs	Autonomous underwater vehicle navigation, habitat monitoring

Key indicator: If the AI’s primary task is to interpret raw input (e.g. image, text, sound, sensor data), this is where it belongs.
Common input types: Visual data (images, video)

Audio and speech

Natural language text

Sensor readings (e.g. haptics, chemical, temperature)

Filter 1

Search titles and descriptions

Sector

Program

Stage

Region

Strategic alignment

AI category

AI sub-category

AI capability

Prime Minister's mandate commitment

Scope

Declared to TBS On hold

Reset

71 current initiative(s)

AI initiative registry
ID	AI Initiative	Stage	Description	Sector	Region	AI Bucket	SME
212	Surveillance électronique en mer Not translated · original: French	Pilot	Il s’agit de capture d’images vidéo sur les navires de pêche pour ensuite les faire analyser par des ressources humaines et un modèle d’inte...	Statistics, Policy and Licensing	Quebec	Perception and Understanding Computer Vision	Rémillard, Vincent
217	Suite du projet de recherche de recherche et développement d’outils de vision automatisée pour l’étude des refuges marins des coraux et des éponges du golfe du Saint-Laurent. Not translated · original: French	Pilot	Une entente de contribution pour l’année 2025-2026 (Programme de gestion des océans) visant à l’amélioration de l’outil développé par le Cen...	Marine Planning and Conservation	Quebec	Perception and Understanding Computer Vision	Brossard, Jade
245	Object, Classification, Tagging, Organization, and Pattern Understanding Solution (OCTOPUS)	Development	This initiative leverages AI technologies to build an Object, Classification, Tagging, Organization, and Pattern Understanding Solution (OC...	Chief Digital Officer	National Capital Region	Perception and Understanding Multimodal AI	Ibrahim, Omar
163	Recherches sur l’Intelligence artificielle pour le traitement automatisé des données d’acoustiques sous-marine Not translated · original: French	Pilot	Nous utilisons les technologies d’apprentissage profond (deep learning) et les détecteurs pour la détection et identification en temps réal...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding Audio Event Detection	Winegardner, Amanda
162	Recherche et développement d’outils de vision automatisée pour l’étude des refuges marins des coraux et des éponges du golfe du Saint-Laurent. Not translated · original: French	Ideation	Projet de contribution avec le Centre de développement et de recherche en intelligence numérique (CDRIN). L’objectif est de concevoir et dév...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding Computer Vision	Winegardner, Amanda
213	Numérisation de documents manuscrits Not translated · original: French	Pilot	En collaboration avec l’équipe nationale de l’Intendant principal des données (PSSI), visant principalement la pêche au saumon. L’objectif e...	Statistics, Policy and Licensing	Quebec	Perception and Understanding Computer Vision	Rémillard, Vincent
218	L’observation des bélugas de l’estuaire du Saint-Laurent Not translated · original: French	Pilot	Entente de contribution avec le Groupe de recherche et d'éducation sur les mammifères marins (GREMM) pour le développement d'un module de me...	Marine Planning and Conservation	Quebec	Perception and Understanding Computer Vision	Robert, Sonia
92	Identification automatisée de la condition de carapace de crabe des neiges Not translated · original: French	Pilot	Détermination de la condition de carapace de crabe des neiges à partir d’images. La méthode offre une façon de valider/améliorer des données...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding Computer Vision	Winegardner, Amanda
223	Automated assessment of satellite imagery	Pilot	AI routines are being developed and tested to assess large volume of satellite imagery and data to: i) Automatically Identify marine hazards...	Ecosystems and Oceans Science	Ontario and Prairie	Perception and Understanding Computer Vision	Klein, David
93	Détection de homards et de crabes communs dans les images sous-marines par intelligence artificielle. Not translated · original: French	Pilot	Développer un processus de capture et d’analyse d’images sous-marine par Intelligence Artificielle pour la détection de homards et de crabes...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding Computer Vision	Winegardner, Amanda
166	Détection de baleines sur les images obtenues par les drones Not translated · original: French	Pilot	Ce projet vise à générer des bases de données d’images annotées et des modèles IA entrainés pour détecter les différentes espèces ou groupes...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding Computer Vision	Winegardner, Amanda
81	Topaz AI Software	Pilot	Enhance underwater video from ROV and tow camera deployments in the offshore region and elsewhere. The deployment of these platforms is incr...	Ecosystems and Oceans Science	Pacific	Perception and Understanding Computer Vision	Best, Merlin
90	Deep Learning Enhances Beluga Whale Research in the Arctic	Pilot	Esri Canada built a deep learning model that automatically detects beluga whales in imagery. The model was developed using satellite, drone...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding Computer Vision	Winegardner, Amanda
76	Chumputer Vision - salmon scale age predictive AI (See B8)	Ideation	Deep Machine Learning and Convolutional Neural Network application for the predictive application of age assignment of Chum salmon by way of...	Ecosystems and Oceans Science	Pacific	Perception and Understanding Computer Vision	Wischniowski, Stephen
74	Pacific Plankton Monitoring: Imagery and automated classification	Pilot	• Monitoring of zooplankton and phytoplankton in the water column with shipboard and benchtop plankton imaging instruments (UVP, ZooScan, Pl...	Ecosystems and Oceans Science	Pacific	Perception and Understanding Computer Vision	Sastri, Akash
72	Extracting impacts data from regulatory documents	Ideation	Use OCR pdf readers, topic detection and language models to detect specific data elements inside written text documents, for extraction and...	Ecosystems and Oceans Science	Pacific	Perception and Understanding Language Understanding (NLP)	Bird, Tom
34	AI-assisted Marine Mammal Annotation Tool	Pilot	The goal of the project is to improve the efficiency of the aerial image annotation process to reduce manual efforts of the marine mammal sc...	Strategic Policy	National Capital Region	Perception and Understanding Computer Vision	Wang, Yuchen
86	Automatic detection of Arctic whales in aerial images	Pilot	In collaboration with Dr. David Clausi and his team at the Vision and Image Processing (VIP) Research Lab (University of Waterloo), we have...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding Computer Vision	Winegardner, Amanda
87	ANMPA Habitat Mapping Project	Pilot	Drop camera survey to document seafloor habitats and biological communities in the Anguniaqvia niqiqyuam Marine Protected Area (ANMPA) in th...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding Computer Vision	Winegardner, Amanda
88	Development of an AI matching model for bowhead whale photo-identification	Pilot	Using DFO’s existing database of bowhead whale photographs, the specific objectives are: 1) Create a machine learning model “detector” for b...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding Computer Vision	Winegardner, Amanda
89	Augmenting Whale Detection in Satellite Images using synthetic data	Pilot	An autodetector model was built to detect whales in satellite imagery using synthetic imagery. The primary author is now testing the algorit...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding Computer Vision	Winegardner, Amanda
95	Optimization of Machine Learning modelling strategy for coastal zooplankton taxonomic identification from flow microscopy imaging.	Pilot	As part of Aquaculture Monitoring Program we are using Ecotaxa's ML algorithm to identify zooplankton. The project aims at identifying the b...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding Computer Vision	Winegardner, Amanda
91	Localization and Tracking of Beluga Whales in Aerial video Using Deep Learning	Pilot	Deep-learning model was developed us YOLOv7 to detect beluga whales from imagery footage. The main contribution of this research is providin...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding Computer Vision	Winegardner, Amanda
94	DFO Bot, deep learning application to analyse otolith images available in DFO Dots	Pilot	Development of a neural network trained using otolith images, expert-derived age estimates, otolith annotations and annuli to obtain age est...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding Computer Vision	Winegardner, Amanda
105	Combining artificial intelligence and drone imagery for automatic extraction of morphometric attributes of small cetaceans: Application to the St. Lawrence Estuary Beluga population.	Pilot	This project proposes to use a multiyear time series of drone images of the St. Lawrence Estuary Beluga population to develop a series of al...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding Computer Vision	Winegardner, Amanda

212
Surveillance électronique en mer
Il s’agit de capture d’images vidéo sur les navires de pêche pour ensuite les faire analyser par des ressources humaines et un modèle d’intelligence artificielle doté de capacité d’identification d’événements, d’espèces, etc. Projet fait en collaboration avec l’équipe nationale de l’Intendant principal des données.

Pilot Statistics, Policy and Licensing Perception and Understanding Computer Vision
217
Suite du projet de recherche de recherche et développement d’outils de vision automatisée pour l’étude des refuges marins des coraux et des éponges du golfe du Saint-Laurent.
Une entente de contribution pour l’année 2025-2026 (Programme de gestion des océans) visant à l’amélioration de l’outil développé par le Centre de développement et de recherche en intelligence numérique (CDRIN) en 2024-2025. Le projet a comme objectif global de concevoir et développer un outil d’intelligence artificielle permettant de détecter et suivre les organismes benthiques filmés dans les refuges marins des coraux et des éponges du Golfe du Saint-Laurent.

Pilot Marine Planning and Conservation Perception and Understanding Computer Vision
245
Object, Classification, Tagging, Organization, and Pattern Understanding Solution (OCTOPUS)
This initiative leverages AI technologies to build an Object, Classification, Tagging, Organization, and Pattern Understanding Solution (OCTOPUS) to enhance the analysis of video, image, and audio data, improve monitoring capabilities and support decision-making processes at DFO. The MVP will initially focus on video processing capabilities, with planned expansion to image and audio analysis in FY 2026–27. In addition, the initiative will deliver integrated data annotation and model training tools to support continuous improvement and reuse of AI models.

Development Chief Digital Officer Perception and Understanding Multimodal AI
163
Recherches sur l’Intelligence artificielle pour le traitement automatisé des données d’acoustiques sous-marine
Nous utilisons les technologies d’apprentissage profond (deep learning) et les détecteurs pour la détection et identification en temps réal des espèces de mammifères marins telles que la baleine noire de l’Atlantique.

Pilot Ecosystems and Oceans Science Perception and Understanding Audio Event Detection
162
Recherche et développement d’outils de vision automatisée pour l’étude des refuges marins des coraux et des éponges du golfe du Saint-Laurent.
Projet de contribution avec le Centre de développement et de recherche en intelligence numérique (CDRIN). L’objectif est de concevoir et développer un outil d’intelligence artificielle permettant de détecter et suivre les organismes benthiques filmés dans refuges marins des coraux et des éponges du Golfe du Saint-Laurent.

Ideation Ecosystems and Oceans Science Perception and Understanding Computer Vision
213
Numérisation de documents manuscrits
En collaboration avec l’équipe nationale de l’Intendant principal des données (PSSI), visant principalement la pêche au saumon. L’objectif est de numériser des documents avec des données manuscrites et de les faire analyser par un modèle d’intelligence artificielle qui va décoder l’écriture pour la transformer en données numériques et les insérer dans une base de données. Le but est de remplacer la saisie manuelle de ces données.

Pilot Statistics, Policy and Licensing Perception and Understanding Computer Vision
218
L’observation des bélugas de l’estuaire du Saint-Laurent
Entente de contribution avec le Groupe de recherche et d'éducation sur les mammifères marins (GREMM) pour le développement d'un module de mesures morphométriques assistées par l’intelligence artificielle pour les bélugas de l'estuaire du Saint-Laurent L’entente s’est terminée le 31 mars 2024 mais le module est en cours d’utilisation. Considérant la contrainte de temps imposée par les mesures manuelles des images prises par drones de bélugas, l’équipe de recherche du GREMM a développé une méthode exploitant l’IA pour mesurer plus rapidement et plus efficacement les bélugas.

Pilot Marine Planning and Conservation Perception and Understanding Computer Vision
92
Identification automatisée de la condition de carapace de crabe des neiges
Détermination de la condition de carapace de crabe des neiges à partir d’images. La méthode offre une façon de valider/améliorer des données de condition de carapace prises sur le terrain. Ces déterminations sont importantes pour quantifier le recrutement à la pêche durant le relevé scientifique ainsi qu’améliorer la qualité des données de crabe mou dans programme observateur-en-mer.

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision
223
Automated assessment of satellite imagery
AI routines are being developed and tested to assess large volume of satellite imagery and data to: i) Automatically Identify marine hazards and ii) Change detection – i.e. shorelines or riverbank changes.

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision
93
Détection de homards et de crabes communs dans les images sous-marines par intelligence artificielle.
Développer un processus de capture et d’analyse d’images sous-marine par Intelligence Artificielle pour la détection de homards et de crabes communs.

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision
166
Détection de baleines sur les images obtenues par les drones
Ce projet vise à générer des bases de données d’images annotées et des modèles IA entrainés pour détecter les différentes espèces ou groupes d’espèces visés.

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision
81
Topaz AI Software
Enhance underwater video from ROV and tow camera deployments in the offshore region and elsewhere. The deployment of these platforms is incredibly time- and resource-consuming, making the optimization of available video an important part of getting the most from this imagery. In addition there are camera limitations from the available DFO survey tools and framegrabs from video will be used as a major product from surveys, making enhancement to high quality images crucial. Topaz Video AI and Topaz Photo AI

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision
90
Deep Learning Enhances Beluga Whale Research in the Arctic
Esri Canada built a deep learning model that automatically detects beluga whales in imagery. The model was developed using satellite, drone and aerial imagery and reported accuracy is high, but the model has not been tested by DFO Science yet.

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision
76
Chumputer Vision - salmon scale age predictive AI (See B8)
Deep Machine Learning and Convolutional Neural Network application for the predictive application of age assignment of Chum salmon by way of scale images. Chum salmon has been selected as the gateway species due to its relatively easy scale pattern interpretation, this will foster the pathway forward for Chinook, Coho Sockeye and Herring as future targets for AI age interpretation. Deep Machine Learning and Convolutional Neural Network within a Python predictive algorithm

Ideation Ecosystems and Oceans Science Perception and Understanding Computer Vision
74
Pacific Plankton Monitoring: Imagery and automated classification
• Monitoring of zooplankton and phytoplankton in the water column with shipboard and benchtop plankton imaging instruments (UVP, ZooScan, PlanktoScope). • Application of automated classification of survey and preserved sample image data sets to complement and enhance spatial and temporal resolution of monitoring in Canada’s EEZ and offshore NE Pacific. Development of and regular optimization of automated image classification models using AI: deep learning (residual networks) and machine learning (random forests) methods.

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision
72
Extracting impacts data from regulatory documents
Use OCR pdf readers, topic detection and language models to detect specific data elements inside written text documents, for extraction and analysis in a cumulative impacts context. Large lanugage models and topic clustering tools

Ideation Ecosystems and Oceans Science Perception and Understanding Language Understanding (NLP)
34
AI-assisted Marine Mammal Annotation Tool
The goal of the project is to improve the efficiency of the aerial image annotation process to reduce manual efforts of the marine mammal science team and reduce time-to-insight for marine mammal survey flights. This project is currently in progress. Business requirements have been finalized and development has commenced Client: Ecosystems and Oceans Science - Ecosystems Science Directorate

Pilot Strategic Policy Perception and Understanding Computer Vision
86
Automatic detection of Arctic whales in aerial images
In collaboration with Dr. David Clausi and his team at the Vision and Image Processing (VIP) Research Lab (University of Waterloo), we have been developing an algorithm to detect narwhals and belugas in photos from aerial surveys. We are planning to expand to ice seals and walrus.

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision
87
ANMPA Habitat Mapping Project
Drop camera survey to document seafloor habitats and biological communities in the Anguniaqvia niqiqyuam Marine Protected Area (ANMPA) in the Inuvialuit Settlement Region. A project sub-objective is to develop AI capacity to detect and record bottom habitat classifications and benthic animals from video still-images, substantially reducing processing times.

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision
88
Development of an AI matching model for bowhead whale photo-identification
Using DFO’s existing database of bowhead whale photographs, the specific objectives are: 1) Create a machine learning model “detector” for bowhead whales. 2) Create a machine learning individual re-ID model for bowhead whales to re-identify bowheads. 3) To make the detector and matching models accessible through an easy to use, freely available, web-based application, Flukebook.org.

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision
89
Augmenting Whale Detection in Satellite Images using synthetic data
An autodetector model was built to detect whales in satellite imagery using synthetic imagery. The primary author is now testing the algorithm on satellite imagery obtained by Arctic Region that has been manually read for whales.

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision
95
Optimization of Machine Learning modelling strategy for coastal zooplankton taxonomic identification from flow microscopy imaging.
As part of Aquaculture Monitoring Program we are using Ecotaxa's ML algorithm to identify zooplankton. The project aims at identifying the best strategy in building model training sets for each region monitored. This strategy will be the basis for future AMP zooplankton identification work.

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision
91
Localization and Tracking of Beluga Whales in Aerial video Using Deep Learning
Deep-learning model was developed us YOLOv7 to detect beluga whales from imagery footage. The main contribution of this research is providing a system that accurately detects and tracks features of beluga whales, both adults and calves, from aerial footage.

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision
94
DFO Bot, deep learning application to analyse otolith images available in DFO Dots
Development of a neural network trained using otolith images, expert-derived age estimates, otolith annotations and annuli to obtain age estimates from otoliths.

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision
105
Combining artificial intelligence and drone imagery for automatic extraction of morphometric attributes of small cetaceans: Application to the St. Lawrence Estuary Beluga population.
This project proposes to use a multiyear time series of drone images of the St. Lawrence Estuary Beluga population to develop a series of algorithms for automatic extraction of morphometric information of individual beluga whales using AI.

Pilot Ecosystems and Oceans Science Perception and Understanding Computer Vision