Visualise networks on map?

[odscjames]

(Possible overlap with #5 . Also see Open-Telecoms-Data/open-fibre-data-standard#124 )

• Agreed on web mercator projection for first pass, in the long run we may change to a different projection
• Choose a mapping library, Leaflet or Open Layers, taking into account the above point
• Choose a base layer provider, taking into account licensing and cost, doesn't need to free
• @duncandewhurst define MVP mapping features, e.g. symbology, categories, layer controls, filters etc.

[Lathrisk]

@duncandewhurst @lgs85 (cc @odscjames)
We have a first pass of some maps in #10

This is now on the dev branch, this URL maybe change in the new future.

I've just set it up with basic circle markers for nodes and a colour for spans. I also, relatively randomly, added some data in a popup for each feature. We can customise all of this as appropriate.

It also uses OSM base layers. This is under an acceptable use policy. We meet the requirements provided we don't err into 'heavy use', I'd guess that OFDS won't, but I could be wrong (and I'd need to check that actually constitutes heavy use). There are other options if we think this is likely, e.g. MapBox, Bing Maps (using a plugin), Esri ArcGIS (official plugin), MapQuest (official plugins) and Here Maps.

Examples here: http://leaflet-extras.github.io/leaflet-providers/preview/

[duncandewhurst]

Looks good! A few questions and suggestions:

• If a network package contains multiple networks, which network is shown on the map?
• If the map only shows one network at a time, remove network from the pop-up.
• Add physical infrastructure provider and network provider to the pop-ups.
• Add a legend (see https://github.com/ptma/Leaflet.Legend)
• Would it be possible to allow users to choose which attributes are used to colour the nodes and spans? I'm thinking something along the lines of:
  • Let users choose from a drop-down of fields that are available in their data for nodes and a drop-down of fields that are available for spans.
  • If a field isn't available for some features, it should still appear in the drop-down. Those features can be assigned to a 'null' category.
  • If a field isn't available for any features, it should be omitted from the drop-down.
  • I think the relevant fields to check are:
    • Nodes:
      • .phase.name
      • .status
      • .type
      • .accessPoint
      • .power
      • .technologies
      • .physicalInfrastructureProvider.name
      • .networkProvider.name
    • Spans:
      • .phase.name
      • .status
      • .physicalInfrastructureProvider.name
      • .networkProvider.name
      • .supplier.name
      • .transmissionMedium
      • .deployment
      • .darkFibre
      • .fibreType
      • .fibreCount
      • .technologies
      • .capacity
  • For fields that are arrays of strings (e.g. .technologies) we can just use the whole array, e.g. 'sdh, dwdm'?
• Add error handling:
  • Don't show the map if there are errors loading nodes.geojson and spans.geojson
  • Report errors to user
  • Colour the visualisation box red if there are errors

[odscjames]

Most of this is for @Lathrisk to tackle but:

If a network package contains multiple networks, which network is shown on the map?

All of them. The JSON->GeoJSON conversion just puts them all together in the 2 files (nodes & spans). If we want the networks split on the map we'll need to add something to that conversion in ODFSKit first. It won't be hard, it just needs prioritising against other work.

Would it be possible to allow users to choose which attributes are used to colour the nodes and spans? .... If a field isn't available for some features ... If a field isn't available for any feature

@Lathrisk we can chat about this one, it might be that the ODFSKit conversion would be a good place to produce a meta file with field use info?

[duncandewhurst]

If a network package contains multiple networks, which network is shown on the map?

All of them. The JSON->GeoJSON conversion just puts them all together in the 2 files (nodes & spans).

I think that's fine. Let's definitely colour the spans and nodes by network by default.

[odscjames]

Ok - @Lathrisk isn't available for a few days but I'll make these decisions:

We don't need to make separate GeoJSON files per network then, and "Let's definitely colour the spans and nodes by network by default." can be handled by the same mechanism as colouring it by other fields.

However it sounds like having meta information available on what fields are used in the GeoJSON would be very handy, so I'll do that.

Thoughts:

.phase.name, .status.name, etc, anything .name - could 2 items have different id's but very similar names or even exactly the same names? There isn't a rule for unique names. Should these ones actually set colours by the id, but show name or some kind of "name (id)" to user?

[duncandewhurst]

We don't need to make separate GeoJSON files per network then, and "Let's definitely colour the spans and nodes by network by default." can be handled by the same mechanism as colouring it by other fields.

👍

However it sounds like having meta information available on what fields are used in the GeoJSON would be very handy, so I'll do that.

In case it's relevant or reusable, Lib CoVE calculates field coverage since OpenDataServices/lib-cove#89.

.phase.name, .status.name, etc, anything .name - could 2 items have different id's but very similar names or even exactly the same names? There isn't a rule for unique names. Should these ones actually set colours by the id, but show name or some kind of "name (id)" to user?

Good point. I've opened an issue on making names unique: Open-Telecoms-Data/open-fibre-data-standard#172. I don't think we want to show ids to users. Setting colours by id but showing names sounds like a reasonable solution. I've added a legend to the list of suggestions in #6 (comment)

[odscjames]

In case it's relevant or reusable, Lib CoVE calculates field coverage since OpenDataServices/lib-cove#89.

Yes - I've just looked at that code as part of doing additional fields - that's why I specially picked this point up :-)

[duncandewhurst]

I've added the following to the list of requirements in #6 (comment):

• Add error handling:
  • Don't show the map if there are errors loading nodes.geojson and spans.geojson
  • Report errors to user with enough information to support troubleshooting
  • Colour the visualisation box red if there are errors

The current feedback to users is contradictory:

[odscjames]

It's possible for the GeoJSON to not actually have any geography elements. In this case, don't show the map. I can add something to Libcoveofds so that the meta file has a flag for this.

In this case, keep the visualisation box and have a message to use "your data has no lat/lngs etc" and put in warning colour - @duncandewhurst will write exact message.

[Lathrisk]

@duncandewhurst @odscjames @codemacabre
There are mixed things going on here. The error messages (There was en error loading:) shown above are generated if there are any errors visualising the data. I think the new message that has been added below those will need to be removed in that instance, regardless of the no lat/lng issue. There could be other reasons the map fails to load.

[duncandewhurst]

I've mapped (no pun intended) out the possible scenarios in the table below - let me know if I missed anything.

What are the possible causes of the There was an error loading: message?

Ideally, the JSON-> GeoJSON conversion script should only produce valid GeoJSON data. Otherwise, if there is an issue with the JSON data that means the GeoJSON would be invalid, then the conversion should fail and report the issue in the data conversion box.

If a user uploads GeoJSON files, are they passed directly to the map or are they first converted to JSON and then back to GeoJSON? If it's the former, then we'll need error handling in the map box too (3rd scenario below).

Scenario	Show map	Message to user	Box colour
GeoJSON contains coordinate data and loads successfully	Yes	The GeoJSON version of your data is visualised on the map below. You should check that nodes and spans appear in the correct location. If not, you should check that your coordinates are in longitude, latitude order. You may need to transform your coordinates to the correct coordinate reference system.	Green (success)
GeoJSON does not contain coordinates	No	Your data cannot be visualised on a map because it does not contain coordinates. If you expected your data to include coordinates, you should check your mapping and data pipeline for errors. For more information, see Geometry	Yellow (warning)
GeoJSON does not load successfully	No	Your data cannot be visualised on a map because there were errors loading it: {{error details}}. You should check your mapping and data pipeline for errors.	Red (Danger)

Note that I updated the successful message slightly to mention coordinate ordering.

[odscjames]

Ideally, the JSON-> GeoJSON conversion script should only produce valid GeoJSON data.

Ideally, but there will be some creative work in the lib to write lots of bad test data then make sure the lib handles it. Open-Telecoms-Data/lib-cove-ofds#27 So for now there could be dodgy GeoJson that crashes our map JS Lib, I think?

If a user uploads GeoJSON files, are they passed directly to the map or are they first converted to JSON and then back to GeoJSON?

The later, and this was something I meant to raise as a general discussion when things are less hectic. #36 for later.

[Lathrisk]

All outstanding comments in this issue are resolved, additional feedback now in : #51