Topics

LayoutEditor refactor, the future of layout configuration information, and the Fall '20 development sequence


Bob Jacobsen
 

Layout Editor was originally an editor for information _about_ _the_ _layout_. The graphics were simple, just sufficient to show connectivity. A single LayoutEditor panel mapped onto information about track, turnouts, signals, etc, and supported tools for configuring them. All that information was stored and loaded via a single <LayoutEditor> element.

As things go in projects like this, a lot was built on top of this: Multiple LayoutEditor panels, much improved graphics, more configuration tools, etc. A lot of people use all those features and like them.

The current master version has about 75k lines (plus tests). About 50 files outside the package refer into it, usually for information about LayoutBlocks or connectivity information. There seems to be a lot of layoutEditor context in the web/ Javascript. We distribute a half-dozen scripts that reference layoutEditor classes, and users have probably written more.

I’ve started a project to:

1) Add a type system, where i.e. a LayoutDoubleSlip (eventually, LayoutDoubleSlipModel and LayoutDoubleSlipView, see below) class implements the code specific to double slips on the layout.

2) Separate the tracking and layout information from its display. From the computing side, this is moving the LayoutTrack and child classes to a MVC model where connectivity ends up in *Model classes, and the panel-specific display information ends up in *View classes.

You can see the progress on this in the bobjacobsen/LE-move-content-MVC* branches and tags. A few of the first bits have been merged into JMRI/master and will be included in 4.20, but the largest changes are waiting to merge until after that.


Currently, the starting point for loading information about e.g. a wye turnout is a `<layoutTurnout>` element:

<layoutturnout ident="TO5" type="WYE_TURNOUT" hidden="no" disabled="no" disableWhenOccupied="no" continuing="2" xcen="200.0" ycen="100.0" xa="216.13423843904454" ya="92.10246276855469" xb="180.0000015154656" yb="100.0" xc="187.7315216064453" yc="115.79507446289062" xd="219.9999984845344" yd="100.0" ver="1" class="jmri.jmrit.display.layoutEditor.configurexml.LayoutWyeXml" turnoutname="IT4" blockname="Yard" connectaname="T9" connectbname="T13" connectcname="T14" />

Most of that information is graphical coordinators for the _panel_, not positions on the layout. (There is currently no place to store layout-specific coordinates) Connectivity is loaded via `<tracksegment>` elements that refer to the ID of the wye, in this case TO5:

<tracksegment ident="T9" blockname="Yard" connect1name="TO5" type1="TURNOUT_A" connect2name="X1" type2="LEVEL_XING_B" dashed="no" mainline="no" hidden="no" class="jmri.jmrit.display.layoutEditor.configurexml.TrackSegmentXml" />

This structure is managed by thousands of lines of code and hundreds of methods in the LayoutEditor, LayoutEditorTools, LayoutEditorAuxTools, LayoutBlockConnectivityTools, ConnectivityUtil classes and others.


I think it would be good to carry the refactor effort further to

3) have a cleaner representation of _just_ the layout with much simpler connectivity mechanisms and including a place to include geometric information about the layout itself if desired,

4) that is separate from the possibility of displaying that in one, two or as many different panels as desired, with different drawings of different parts of the layout.

As a step toward that, the refactor branch has moved the repository of the LayoutTrack(Model) classes to an (incomplete, but working) LayoutModels repository class that is _separate_ from the existing LayoutEditor. There’s also a separate LayoutEditorViewContext class to properly store global graphical defaults away from the LayoutEditor editor class. The goal is for LayoutEditor to be only a representation of one-of-many panels, removing the remnants of it being a single repository of all layout information.

I want to stress that I think David Duchamp did a great thing in creating LayoutEditor, and that more than a dozen people have made valuable contributions to it since then.

The question is whether this Fall is a good time to move to a more powerful internal model, one that could hardly have been imagined when Layout Editor was first created in 2004.

To really do (3) and (4) involves not just cleaning up the existing classes, but creating a better underlying representation of the layout itself. (That could either be a native replacement for the LayoutEditor classes that do that now, or there could be a translation step) Part of the reason for the large number of ’tools’ routines is the difficult navigational structure of the existing classes.


Is this Fall the right time to do this renovation? Are there other big projects being planned for the next development series?

Bob

--
Bob Jacobsen
@BobJacobsen


Randall Wood <rhwood@...>
 

I think that Fall is the right time to do this renovation, but I also think we need to be thinking about:

- The timeline for requiring Java 11
- Java 8 updates from Oracle ceases to be publicly available in December 2020
- We are relying on the use of scripts to allow compilation on Java 8 and running on Java 9+ in macOS — I’d like to drop all the complexity and fragility that encompasses

I do have a project that I am noodling on, that will have an effect on the underlying model for representing the layout. I’ll describe the desired effect, since I haven’t really gotten beyond that:
- Given a modular railroad (say an NTRAK layout) where modules (or sets of modules) have an embedded JMRI processor in them, it would be good to be able to have these processors operate in a JMRI mesh where it is easy to order panels together on the fly (these modules will not necessarily always operate together or be assembled in the same order).

I think this implies that JMRI needs to be able to keep track of where a NamedBean is defined (and how it was defined), where it is used (and how it is used) and possibly allow that a single device on a layout may have multiple NamedBean identities associated with it (and how to map those identities to a single processing NamedBean). I don’t know if or that this implies that JMRI would not, when storing panels, write all NamedBeans.

On 31-May-2020, at 21:51, Bob Jacobsen <@BobJacobsen> wrote:

Layout Editor was originally an editor for information _about_ _the_ _layout_. The graphics were simple, just sufficient to show connectivity. A single LayoutEditor panel mapped onto information about track, turnouts, signals, etc, and supported tools for configuring them. All that information was stored and loaded via a single <LayoutEditor> element.

As things go in projects like this, a lot was built on top of this: Multiple LayoutEditor panels, much improved graphics, more configuration tools, etc. A lot of people use all those features and like them.

The current master version has about 75k lines (plus tests). About 50 files outside the package refer into it, usually for information about LayoutBlocks or connectivity information. There seems to be a lot of layoutEditor context in the web/ Javascript. We distribute a half-dozen scripts that reference layoutEditor classes, and users have probably written more.

I’ve started a project to:

1) Add a type system, where i.e. a LayoutDoubleSlip (eventually, LayoutDoubleSlipModel and LayoutDoubleSlipView, see below) class implements the code specific to double slips on the layout.

2) Separate the tracking and layout information from its display. From the computing side, this is moving the LayoutTrack and child classes to a MVC model where connectivity ends up in *Model classes, and the panel-specific display information ends up in *View classes.

You can see the progress on this in the bobjacobsen/LE-move-content-MVC* branches and tags. A few of the first bits have been merged into JMRI/master and will be included in 4.20, but the largest changes are waiting to merge until after that.


Currently, the starting point for loading information about e.g. a wye turnout is a `<layoutTurnout>` element:

<layoutturnout ident="TO5" type="WYE_TURNOUT" hidden="no" disabled="no" disableWhenOccupied="no" continuing="2" xcen="200.0" ycen="100.0" xa="216.13423843904454" ya="92.10246276855469" xb="180.0000015154656" yb="100.0" xc="187.7315216064453" yc="115.79507446289062" xd="219.9999984845344" yd="100.0" ver="1" class="jmri.jmrit.display.layoutEditor.configurexml.LayoutWyeXml" turnoutname="IT4" blockname="Yard" connectaname="T9" connectbname="T13" connectcname="T14" />

Most of that information is graphical coordinators for the _panel_, not positions on the layout. (There is currently no place to store layout-specific coordinates) Connectivity is loaded via `<tracksegment>` elements that refer to the ID of the wye, in this case TO5:

<tracksegment ident="T9" blockname="Yard" connect1name="TO5" type1="TURNOUT_A" connect2name="X1" type2="LEVEL_XING_B" dashed="no" mainline="no" hidden="no" class="jmri.jmrit.display.layoutEditor.configurexml.TrackSegmentXml" />

This structure is managed by thousands of lines of code and hundreds of methods in the LayoutEditor, LayoutEditorTools, LayoutEditorAuxTools, LayoutBlockConnectivityTools, ConnectivityUtil classes and others.


I think it would be good to carry the refactor effort further to

3) have a cleaner representation of _just_ the layout with much simpler connectivity mechanisms and including a place to include geometric information about the layout itself if desired,

4) that is separate from the possibility of displaying that in one, two or as many different panels as desired, with different drawings of different parts of the layout.

As a step toward that, the refactor branch has moved the repository of the LayoutTrack(Model) classes to an (incomplete, but working) LayoutModels repository class that is _separate_ from the existing LayoutEditor. There’s also a separate LayoutEditorViewContext class to properly store global graphical defaults away from the LayoutEditor editor class. The goal is for LayoutEditor to be only a representation of one-of-many panels, removing the remnants of it being a single repository of all layout information.

I want to stress that I think David Duchamp did a great thing in creating LayoutEditor, and that more than a dozen people have made valuable contributions to it since then.

The question is whether this Fall is a good time to move to a more powerful internal model, one that could hardly have been imagined when Layout Editor was first created in 2004.

To really do (3) and (4) involves not just cleaning up the existing classes, but creating a better underlying representation of the layout itself. (That could either be a native replacement for the LayoutEditor classes that do that now, or there could be a translation step) Part of the reason for the large number of ’tools’ routines is the difficult navigational structure of the existing classes.


Is this Fall the right time to do this renovation? Are there other big projects being planned for the next development series?

Bob

--
Bob Jacobsen
@BobJacobsen






Paul Bender
 

On Jun 2, 2020, at 7:20 AM, Randall Wood via groups.io <rhwood=mac.com@groups.io> wrote:
I do have a project that I am noodling on, that will have an effect on the underlying model for representing the layout. I’ll describe the desired effect, since I haven’t really gotten beyond that:
- Given a modular railroad (say an NTRAK layout) where modules (or sets of modules) have an embedded JMRI processor in them, it would be good to be able to have these processors operate in a JMRI mesh where it is easy to order panels together on the fly (these modules will not necessarily always operate together or be assembled in the same order).
This is something I have long wished for as well. This is part of the reason:
1) Layout Editor Panels have edge connectors
2) The JMRI Network Client allows you to alias the remote connections.

My thought is that the module attached processor is something along the lines of a RaspberryPi. I already have modules with such devices attached.

I think this implies that JMRI needs to be able to keep track of where a NamedBean is defined (and how it was defined), where it is used (and how it is used) and possibly allow that a single device on a layout may have multiple NamedBean identities associated with it (and how to map those identities to a single processing NamedBean). I don’t know if or that this implies that JMRI would not, when storing panels, write all NamedBeans.
If you really are thinking a mesh, then there are several technical issues to consider.

It is easiest to think about this from the point of view of a single network node talking to its peers, but here are a few of the questions that need to think about:
1) each node needs to determine if it consumes the information or if it just passes it along. We can’t assume that any node in the network can hear the whole network.
2) is the connected node’s resource connected only to that node or is it connected to multiple nodes? ( I.e. is it connected to a RaspberryPi GPIO pin or is it connected to LocoNet)
3) Is JMRI routing messages, or are we going to use a service to do that for us?
4) can the network topology change on the fly? This may or may not be something that happens often, but it can happen. Nodes can go on and offline for various reasons. New segments can be added to the railroad ( this one easier to think about with a linear modular format ( like Fremo/Free-mo/Free-moN ) than an oval format ( like N-trak).

And then there are the addressing concerns for global resources, but that is not really a JMRI concern. ( I.e. two modules not have an accessory decoder set to the same address).

More thoughts on this later.

Paul


Bob Jacobsen
 

I don’t think layoutEditor edge connectors (a specialization of PositionablePoint), as they now exist, are likely to survive any refactoring to a common model.

They exist because the existing LayoutEditor (the panel class) objects contain the layout track items, and that’s a one-to-many-exclusive relation. When the possibility of have two LayoutEditor (panel) objects was added, they had to relate their non-overlapping internal models to each other. That was done through edge connectors.

That was probably a rational step, no criticism of that decision to get something done, but (a) it makes the code significantly more complex to be able to handle that partition and (b) there’s no need for edge connectors when you have multiple panels working off a single model.

I’m not sure what they right use cases and internal representations are for modular railroads. We should work through that. But I doubt that the existing edge connector tech is the answer.

Bob

On Jun 2, 2020, at 10:45 AM, Paul Bender <paul.bender@...> wrote:

1) Layout Editor Panels have edge connectors
--
Bob Jacobsen
@BobJacobsen


Dave Sand
 

I don't know if it is proper to insert the following into this thread, but it seems somewhat relevant.

The attached image shows the Layout Editor initialization sequence for track connectivity, block paths and advanced block routing.

The advanced routing section is sparse since I could not figure out how to represent how it actually works. Each layout block has 0 to N neighbors. The neighbors talk to each other using PropertyChangeEvents. Depending on the event type and content, the events are propagated to the recipient's neighbors. Eventually every layout block ends up with a table of every other layout block and the number of hops to get there.

While the block paths are derived from LE panel connectivity, they represent a single model. And advanced block routing is built on the paths which makes it a single model.


Dave Sand

----- Original message -----
From: Bob Jacobsen <@BobJacobsen>
To: jmri@jmri-developers.groups.io
Subject: Re: [jmri-developers] LayoutEditor refactor, the future of layout configuration information, and the Fall '20 development sequence
Date: Tuesday, June 02, 2020 1:02 PM

I don’t think layoutEditor edge connectors (a specialization of PositionablePoint), as they now exist, are likely to survive any refactoring to a common model.

They exist because the existing LayoutEditor (the panel class) objects contain the layout track items, and that’s a one-to-many-exclusive relation. When the possibility of have two LayoutEditor (panel) objects was added, they had to relate their non-overlapping internal models to each other. That was done through edge connectors.

That was probably a rational step, no criticism of that decision to get something done, but (a) it makes the code significantly more complex to be able to handle that partition and (b) there’s no need for edge connectors when you have multiple panels working off a single model.

I’m not sure what they right use cases and internal representations are for modular railroads. We should work through that. But I doubt that the existing edge connector tech is the answer.

Bob

On Jun 2, 2020, at 10:45 AM, Paul Bender <paul.bender@...> wrote:

1) Layout Editor Panels have edge connectors
--
Bob Jacobsen
@BobJacobsen


Bob Jacobsen
 

Thanks! That’s a very useful diagram.

Can I add it to the package documentation?

A more basic question: Is there a reason that these connections have to be created dynamically rather than being stored and reloaded?

Bob

On Jun 2, 2020, at 1:35 PM, Dave Sand <@davesand> wrote:

I don't know if it is proper to insert the following into this thread, but it seems somewhat relevant.

The attached image shows the Layout Editor initialization sequence for track connectivity, block paths and advanced block routing.

The advanced routing section is sparse since I could not figure out how to represent how it actually works. Each layout block has 0 to N neighbors. The neighbors talk to each other using PropertyChangeEvents. Depending on the event type and content, the events are propagated to the recipient's neighbors. Eventually every layout block ends up with a table of every other layout block and the number of hops to get there.

While the block paths are derived from LE panel connectivity, they represent a single model. And advanced block routing is built on the paths which makes it a single model.


Dave Sand
--
Bob Jacobsen
@BobJacobsen


Dave Sand
 

Bob,

Yes, please add it. I will attached the text file.

The block paths are stored and reloaded, but they are verified and updated after panel loading. The block table has an option to delete the paths before storing . This results in a complete re-build of the paths. This is sometimes necessary when making a lot of block changes on a panel. There appear to be edge cases that are not handled properly by the change induced updates.

For advanced block routing there is a volume issue. A hundred layout blocks each with a list of 99 destination blocks makes for a lot of data. This also provides recovery from errors introduced by edge cases.

Dave Sand

----- Original message -----
From: Bob Jacobsen <@BobJacobsen>
To: jmri@jmri-developers.groups.io
Subject: Re: [jmri-developers] LayoutEditor refactor, the future of layout configuration information, and the Fall '20 development sequence
Date: Tuesday, June 02, 2020 3:55 PM

Thanks! That’s a very useful diagram.

Can I add it to the package documentation?

A more basic question: Is there a reason that these connections have to be created dynamically rather than being stored and reloaded?

Bob

On Jun 2, 2020, at 1:35 PM, Dave Sand <@davesand> wrote:

I don't know if it is proper to insert the following into this thread, but it seems somewhat relevant.

The attached image shows the Layout Editor initialization sequence for track connectivity, block paths and advanced block routing.

The advanced routing section is sparse since I could not figure out how to represent how it actually works. Each layout block has 0 to N neighbors. The neighbors talk to each other using PropertyChangeEvents. Depending on the event type and content, the events are propagated to the recipient's neighbors. Eventually every layout block ends up with a table of every other layout block and the number of hops to get there.

While the block paths are derived from LE panel connectivity, they represent a single model. And advanced block routing is built on the paths which makes it a single model.


Dave Sand
--
Bob Jacobsen
@BobJacobsen


Paul Bender
 

On Jun 2, 2020, at 2:02 PM, Bob Jacobsen <@BobJacobsen> wrote:

I don’t think layoutEditor edge connectors (a specialization of PositionablePoint), as they now exist, are likely to survive any refactoring to a common model.

They exist because the existing LayoutEditor (the panel class) objects contain the layout track items, and that’s a one-to-many-exclusive relation. When the possibility of have two LayoutEditor (panel) objects was added, they had to relate their non-overlapping internal models to each other. That was done through edge connectors.
One of the original use cases for the edge connectors was for modular layouts.

The idea was that individual module owners could build a panel for their modules. When it was time to join modules together, you sent your panels to the layout coordinator and they would build a combined panel for the layout. All it takes to build the layis for the layout coordinator to stitch the modules together using the edge connectors.

There may be other use cases as well, but this is a valid use case that needs to be reflected in the logical model. In other words, you have to be able to model a module, which could be used independently, but more often will be used with other panels that were created by someone else.

Paul


Petr Šídlo
 

User wish list

Today's computer-controlled interlocks system have rich options for displaying the condition of the layout. The current Layout Editor only supports 3 track states. It would be good to extend these options by at least one frequently used state.
In this example (from STARMON - Czech company):
gray - unoccupied block
red - occupied block
green - allocated for the train
white - allocated for shunting - new option



--
Petr Šídlo
Czech Republic
https://sites.google.com/site/sidloweb/