Challenge #5: Mid-term Report

Since the inception of map whiteboard challenge and the first draft release in summer, various improvements have been made to the software parts enabling whiteboard functionality. This document will summarize the developments since that time. The current working prototype of the integrated system runs at https://dih.bosc.lv/modeller/private/ and is available for authorized users. The system’s purpose is to provide the user the possibility to create system dynamics models with linked spatial information in a form of map composition for each graph or model.

Whiteboard library and server configuration

Server architecture

A docker based environment has been setup to run the components in production (see the diagram).

It consists of 2 mongo databases – one for the container application (Polirural modeller) and one for map whiteboard. Similarly we use 2 rest api endpoints named polirural-model-builder-api and pmb-whiteboard-server where the whiteboard-server supports both REST and Websocket protocols. At the top resides an nginx proxy to rule them all. 

The map whiteboard server connects to model-builder-api provided GRAPHQL endpoint to validate user sessions and get the current users info. This is used as an alternative to OAUTH2, but conceptually does the same thing.

Client architecture

On the client side we have an Angular9 based application, which incorporates a map window component based on HSLayers-ng. Since it runs Openlayers under the hood, we can integrate the Map whiteboard library using the underlying Openlayers map object.

Since Hslayers already provides feature drawing functionality we have added to map whiteboard library the possibility to turn off the drawing toolbar not to have duplicate user controls and call the whiteboard libs data synchronization methods directly from javascript using hooks in hslayers. Also switching the current edited layer on mapwhiteboard library from outside needed to be created since the application supports multiple editable vector layers which are done on hslayers side. 

Map composition preparation

The map composition creation is still done in the application outside map whiteboard lib with the help of hslayers-ng, because it supports multiple layer types and parameters. The json serialize map object or individual layer is then passed to map whiteboard client side code, which then sends it to the server. This happens at the moment when the user selects a ‘Model’ in the dropdown list on the top-left corner of the UI. The software then checks if a map composition is linked to the model and creates it in case it doesn’t exist. In case it already exists the composition is being queried from the map whiteboard server, and layers are populated on the map including the features. 

Recently we also developed an initial user rights model to keep track of who can access and edit the compositions.

The uploaded composition is stored on the harddrive, parsed and the title, description and filename of the composition is stored in Mongo database.

Downloading map composition file was developed (currently without embedded features)

Currently functionality for editing the composition supports:

  • Add new layers either drawable vector or wms, arcgis or vector data loaded from external files
  • Listing of editable layers parsed from composition file
  • Requesting features for particular layers. Multiple editable layers are now supported
  • Listing of my uploaded compositions for current user
  • Granting access (owning) rights to the uploader of composition.

Still needs developing:

  • Granting and removing access rights to additional users. Currently all users can see and edit the composition only they don’t appear in the list of compositions he/she owns.
  • Renaming of layers.
  • Deleting of composition (available to owner).
  • Styling of features
  • Activation of scratch layer
  • Serialization of layers and composition on the map-whiteboard side. 

Features

Whenever a user draws a feature, they are propagated to all the clients currently using the same composition meaning they selected the same ‘Model’ in the dropdown. Features are now linked to particular composition and layer, where previously we had a single fixed editable layer per composition. The features are stored in mongo databases  and modification of them is done through CRUD operations on the Websockets API.

Listing of features is done for a specific editable layer by its title through REST API.

Challenge #4: Mid-term Report

The report summarizes the outputs achieved in Challenge #4 of the COVID-19 INSPIRE Hackathon during the first part of the event.

  • Number of registered participants: 28
  • Challenge #1 Shared Workspace (complete information about activities in the challenge)
  • Interconnection with pilot activities of Polirural project → 15 of the registered participants are involved in Polirural pilots and represent 8 of the 12 pilot regions

  • Webinar – October 12, invitation post, presentation, 27 live attendees
  • Development of testing scenarios 
  • Clustering – testing various clustering methods, imputation of missing data and combination of non-hierarchical and hierarchical clustering

Challenge #3: Mid-term Report

This mid-term report of Challenge #3 contains the most interesting and valuable achievements, which have been done so far.

  • There are 19 registered participants in our challenge.
  • A collaboration with challenges has been established or being negotiated:
  • CHALLENGE #8 Digitalization of indigenous knowledge in African agriculture for fostering food security – through the incorporation into the African community around such a challenge, 
  • CHALLENGE #13 Calculation of agro-climatic factors – potential source of information for forecasting regional food supplies – through the visualization and map sharing of calculated agro-climatic factors (ongoing),
  • CHALLENGE #4 Rural Attractiveness Visualization – through the sharing of maps created in this challenge (negotiated).
  • Our intention is based on map data sharing using three main open source components (see the figure below representing a schema of map data publishing):
    • Layer Manager (Layman) QGIS Plugin
    • HSLayers-NG
    • Digital Innovation Hub (particularly SmartAfriHub)

Thanks to the intensive use of the mentioned components, there is a rapid development of them, fixing bugs appeared, enhancing user experience and spreading information about such tools for map data publishing among users.

    • Solved issues regarding authentication of QGIS plugin
    • Layman plugin installer updated to be more intuitive
    • Plugin refactored to request Layman asynchronously
    • Czech localization added to the plugin
    • Resolved sharing from HSLayers client to social networks
    • HSLayers Layman errors notifications made more user friendly
    • Fixed bugs and adapted GUI of HSLayers drawing tool based on user feedback
    • Another minor bug fixes
  • Creation of Covid-19 related maps and map compositions. See the examples below:
    • Map composition of Covid-19 cases and deaths (new and cumulative) on 1st October 2020 worldwide displayed in QGIS SW

  • Map composition of Covid-19 cumulative cases from the previous example displayed in the HSLayers web client, i.e. map composition is stored on a HUB and visualized using a web browser

  • Map composition on Covid-19 weekly cases/cured/deceased in the Czech Republic from 30th September (ongoing work)

  • Components for metadata harvesting have been set up in Micka catalogue and tested (https://www.agrihub.cz/micka/). Map compositions and data layers had been harvested from the Czech National GeoPortal (https://geoportal.gov.cz/)
  • A webinar covering Challenge 3 topics was broadcasted on the 1st of October, the webinar’s recording is available here: https://event.webinarjam.com/replay/59/nkpy0i5pumzcvyur50 
  • Step-by-step video sequences on how to use Layman QGIS plugin (available here: https://youtu.be/8Z1Gube6nuc) for map compositions and how to use HSLayers-NG for online creation and visualization of map compositions (available here: https://youtu.be/JFAhBCYPaSU) were shooted
  • Future plans:
    • Further development of the mentioned involved tools
    • Further incorporation of users in development of the tools
    • Spreading the information about our tools and our open source-based approach
    • Sharing more maps and information helping to established Citizens Science Network for Peer to Peer Maps Sharing
    • Fulfill cooperation with other challenges of Covid-19 INSPIRE Hackathon

Challenge #2: Mid-term Report

The report summarizes the outputs achieved in Challenge #2 of the COVID-19 INSPIRE Hackathon during the first part of the event.

  • Number of registered participants: 11
  • Challenge # 2 Shared Worskpace 
  • The final goals of the challenge:
    • Main goal:
      • To develop the advanced and much more practical version of Atlas of Regional Specialities and the e-shop and to think out the proper way to intertwine it finally and usefully to successfully promote both but especially the region itself.
      • The challenge aims to eliminate the above-mentioned problem situations by supporting regional primary producers and primary food processors (farmers, fruit and vegetable growers, butchers, etc.), collecting information about their products and making this information and products available to end customers.
    • Secondary goals
      • To present midterm results in the events organized in the hackathon.
      • To create an active group of co-workers (developers, testers, data providers, feedback providers, etc.).
      • To implement a special set of functionalities for further innovation of the Atlas.
      • To prepare a space for possible future steps on replication of such Atlas also for further regions with a special and similar local-regional specialities. 
      • To figure out and arrange the sustainability of the solution.
      • To promote the H2020 project LIVERUR and vice versa as the background for the e-commerce side of the Atlas being developed in cooperation between 2 Czech partners (one technical, second one pilot): WRLS and UHLAVA. 
  • Brainstorming session on Jamboard during the webinar (20/10/2020) via QR code:

Background:

Problem situations related to Covid-19, such as restrictions or temporary interruptions of work, quarantine, restrictions on the number of people in stores or restrictions on sales time, can lead to a change in shopping habits. The result can be an oversupply of households leading to unequal distribution, a temporary shortage of certain foods, and rising prices.

The challenge proposal aims to eliminate the above-mentioned problem situations by supporting regional primary producers and primary food processors (farmers, fruit and vegetable growers, butchers, etc.), collecting information about their products and making this information and products available to end customers.

Conception:

One of the possible solutions is to connect regional information about farmers and their available products with various platforms for trading in these products (stone shop, e-shop, purchase from the yard). The Atlas of the Best Practices will be used as the basic technological platform, on the basis of which a pilot prototype of the Atlas of Regional Specialties will be created.

Atlas of Regional Specialities

 

Challenge #1: Mid-term Report

The report summarizes the outputs achieved in Challenge #1 of the COVID-19 INSPIRE Hackathon during the first part of the event.

  • Challenge #1 Shared Workspace (complete information about activities in the challenge)
  • The final goals of the challenge:
    • Main goal:
      • To develop the advanced catalogue of educational materials (best practices, educational documents, application, services, other resources…) suitable for remote teaching/learning in the EBAG domain.
    • Secondary goals
      • To present results in the events organized in the hackathon.
      • To create an active group of co-workers (developers, testers, data providers, feedback providers, etc.).
      • To implement advanced techniques (such as searching similarities through clusters, links to controlled vocabularies, etc.) to enrich the catalogue.
      • To interconnected with contemporary trends such as Responsible Research and Innovation, FAIR data (findability, accessibility, interoperability, and reusability), Linked Open Data or international standards.
      • To figure out and arrange the sustainability of the solution.
  • Brainstorming session on Jamboard

  • Catalogue for EBAG Education

    • Table with data – the current version contains 65 items from 3 providers (Czechia, Tunisia)
    • The data model of the catalogue based on Dublin Core and other standards, respecting Linked Open Data approach and Responsible Research and Innovation principles

Initial attempts with categorized data (metadata) clustering for searching homogeneities and similarities

EUXDAT Webinar Series – Pilot’s Session

Did you miss another webinar held in the frame of EUXDAT Webinar Series? No worries, we have a recording for you so that you can watch it now!

Please, helps us to get your opinion on the EUXDAT project.by filling in the questionnare.

Webinar Agenda:

EUXDAT relevance towards Farm to Fork and Towards a Green Cap. Karel Charvát (CoO)
The Farm to Fork Strategy, s to address the challenges and accelerate the transition to sustainable food systems, to ensure that the economic, social and environmental foundations of food and nutrition security are not compromised for current and future generations. To be possible implement challenging goal of Farm Fork Strategy, there will be necessary to introduce effective knowledge management. We will share our experience, how new technologies can help with implementation of such knowledge management system.

Crop Climate Risk Analysis, Monitoring, and Prediction. Nicoletta Addimando, Christoph Ramshorn (meteoblue)
We combine local climate data with information on a crop’s susceptibility to drought and frost in different growth stages. This allows us to analyse which crops are best suited for a location as it is affected by climate change. We can also monitor crop weather risks as the current growing season unfolds and predict crop weather risks for 5 to 14 days.

Discussion and feedback analysis, Karel Charvát (CoO), Jorge Lopez (ATOS)

Invitation to the Webinar on Business Plans

Register now!

The COVID-19 INSPIRE Hackathon 2020 Webinar Series reaches its end already on Monday 26th October with the Webinar on Introduction to Business Plans.
Most people want to start their business just by jumping in and doing «it». If they would dedicate some time to the planning and forming a sensible business plan, they would save both time and money. So let’s take a look at how they are formed in ESA BIC Prague. 

Do you want to learn How to formulate a clear story in your business plan? What are the most common mistakes that you can avoid? How to write your business plan for you and the reader? Then do not hesitate to register!

Register now!

Agenda:

  • What is the Business plan?
  • Lean canvas
  • TRL
  • SWAT an risks
  • Market analysis
  • Gantt chart
  • Conclusion

You will hear from

 

Final Webinars of the EUXDAT Project “The EUXDAT e-Infrastructure”

Yesterday, a final webinar series of the EUXDAT project started with the webinar dedicated to The EUXDAT e-Infrastructure. In case you missed it, we have its recording so you can watch it now!

Please, helps us by filling in the questionnare to get your opinion on the EUXDAT project.

The agenda of the webinar:

Introduction of Series EUXDAT Webinars. Karel Charvát (CoO)

Introduction to series of Webinars, purpose of  Webinars and introduction of EUXDAT questionnaire

Green Deal, Destination Earth and EUXDAT. Nieto De Santos, Francisco Javier (ATOS)

This webinar will describe the main features of the EUXDAT e-Infrastructure and will analyse the main topics defined in the last initiative of the EC, focusing on the Green Deal. We will explain how EUXDAT can contribute to the EC vision and which features might be used by researchers and practitioners, in order to facilitate the implementation of the proposed ideas and concepts

The EUXDAT End user Platform. Anne Haugommard, Fabien Castel (ATOS France)

In this session we will present the EUXDAT platform features offered in the web platform. An overview of the platform components and hosting e-infrastructure architecture, the data catalogue, the development environment with notebooks to develop new value-added algorithms using data connectors and geoprocessing libraries.

The orchestrator, parallelization, monitoring, and accounting in EUXDAT. Jesus Gorroñogoitia Cruz,  Paolo Marangio (ATOS Spain)

This webinar presents the architecture implemented for managing the underlying infrastructure of EUXDAT, so it is possible to use Cloud and HPC resources in the Agriculture domain easily. It will present the different components involved (orchestrator, monitoring, accounting, Service Level Agreements manager) and how they are connected. In the webinar we will explain the features provided and how they can be used. We will also explain how we parallelized one of the EUXDAT scenarios and how it is possible to run it through the described components.

Parallelizing with Python-MPI. Jose M. Montañana (HLRS)

The amount of data available for Geoprocessing applications has been increasing exponentially during the last decades, as well as the size of the problems to solve, which requires increasing the computation capabilities. In order to achieve it and reduce computation time, supporting tools have been developed to ease the distribution of tasks among multiple processing nodes. In this session, we focus on an introduction to how to use Python-MPI. It allows parallelizing existing applications for its execution on HPC and on Cloud.

Discussion and feedback analysis. Karel Charvát (CoO), Jorge Lopez (ATOS)

 

Webinar on A Map WhiteBoard for Collaborative Map-Making

Let us cordially invite you to the webinar, where the “Map Whiteboard for Collaborative Map-Making and Data Capture” will be presented. The working title of their effort is a “Map Whiteboard” and the transport and storage for data is a JSON based format called “Map Compositions” that draws on the early work of Web Map Context documents, but that extends them with 20 years worth of added features. This webinar related to challenge No. 5 of COVID-19 INSPIRE Hackathon 2020 is scheduled on 22nd October at 1PM CEST. Join the webinar to learn more about this extremely useful tool that is being developed in collaboration with BOSC and AVINET. Do you want to learn more? Then do not hesitate to join us on Thursday!

Register now!

Agenda of the webinar:

  • Map Compositions format
  • Collaborative Map-Making – Introduction to map compositions, Example of composition structure
  • Introduction to map WhiteBoards
    • Authorization and ownership
    • Layer management
    • Feature synchronization

You will hear from 

Final EO4Agri Workshop

This is the final workshop of the EO4AGRI project, which is planned as an open meeting to present the final EO4AGRI Strategic Research Agenda and recommended Roadmap towards improving the European capacity for operational agriculture monitoring based on information derived from Copernicus satellite observation data and through the exploitation of associated geospatial and socio-economic information services.

The workshop will be split into two sessions of around 1.5 hours each. The first section is oriented to researchers, developers, scientific communities, etc, and will be focused on the methodology, user requirements, and gap analysis processes developed during the project in order to get the results that are gathered in the EO4AGRI Strategic Research Agenda and Roadmap that will be presented during the second session (which is mainly targeted to policymakers, public administrations, etc).

Additionally, each slot will be combined with a Coffee table where participants will be able to interact with the speaker and discuss hot topics of that slot.

AGENDA

28/10/2020 || 14:00 CET : Technical insights on the use of Earth observation for agricultural applications

  1. Overall event concept
  2. EO4Agri – Project overview
  3. User Requirements and Gap Analysis for the use of Earth observation data in agricultures
  4. Technical landscape
  5. Methods for Agriculture Applications
  6. Infrastructure & Software
  7. EO4AGRI catalogue.
  8. Q&A

29/10/2020 || 14:00 CET: EO4AGRI Strategic Research Agenda and Roadmap

  1. Overall event concept.
  2. Project Introduction
  3. Earth Observation for Agriculture Strategy and Implementation of the Green Deal
  4. EO4AGRI Strategic Research Agenda for Earth Observation in Agriculture in  Europe beyond 2020
  5. EO4AGRI Policy Roadmap – concise guidelines for EO in agriculture for CAP post-2020
  6. Q&A