Topics

Time to retire 'Store Configuration Only to File'?


Matthew Harris
 

Dear all,

Given the myriad of confusion surrounding the whole panel storage options - see recent jmriusers thread https://groups.io/g/jmriusers/topic/76465874 as an example of this - is it time for us to consider removing the 'Store Configuration Only' option?

I'm just trying to understand the use cases for such an option - which scenarios can benefit from having the ability to save everything aside from the panel editor details?

The whole story around how we persist 'state' between JMRI sessions seems to regularly generate questions and confusion within the user community.

Thoughts?

Best regards,

Matt H


Bob Jacobsen
 

Historically, it was introduced as a migration tool. I’m note sure it has any use now that we have better tools for moving user names, etc. Maybe leave it in the Debug menu?

We could also consider a “store a file on every quit” i.e. YourLastSession.xml”, maybe as an option.

And perhaps simpler ways of setting up the usual “load at startup” paradigm.

Bob

On Aug 28, 2020, at 11:03 AM, Matthew Harris <@mattharris> wrote:

Dear all,

Given the myriad of confusion surrounding the whole panel storage options - see recent jmriusers thread https://groups.io/g/jmriusers/topic/76465874 as an example of this - is it time for us to consider removing the 'Store Configuration Only' option?

I'm just trying to understand the use cases for such an option - which scenarios can benefit from having the ability to save everything aside from the panel editor details?

The whole story around how we persist 'state' between JMRI sessions seems to regularly generate questions and confusion within the user community.

Thoughts?

Best regards,

Matt H

Bob Jacobsen
@BobJacobsen


whmvd
 

"We could also consider a “store a file on every quit” i.e. YourLastSession.xml”, maybe as an option.

And perhaps simpler ways of setting up the usual “load at startup” paradigm."

I learnt today that closing a session does not necessarily generate the 'do you want to save your changes?' pop-up. The generation of a YourLastSession.xml would, I take it, be an extra action wherever that pop-up is generated, and that would make it similarly unreliable. Otherwise, I'd be all for it. Especially as then you wouldn't need to do anything to the "load at startup", because loading it at start-up with "YourLastSession.xml" would then allow the vast majority of people to forget about the whole store/load business. Sounds pretty ideal, in fact!

Wouter


On Fri, 28 Aug 2020 at 16:52, Bob Jacobsen <rgj1927@...> wrote:
Historically, it was introduced as a migration tool.  I’m note sure it has any use now that we have better tools for moving user names, etc.  Maybe leave it in the Debug menu?

We could also consider a “store a file on every quit” i.e. YourLastSession.xml”, maybe as an option.

And perhaps simpler ways of setting up the usual “load at startup” paradigm.

Bob

> On Aug 28, 2020, at 11:03 AM, Matthew Harris <matthew.john.harris@...> wrote:
>
> Dear all,
>
> Given the myriad of confusion surrounding the whole panel storage options - see recent jmriusers thread https://groups.io/g/jmriusers/topic/76465874 as an example of this - is it time for us to consider removing the 'Store Configuration Only' option?
>
> I'm just trying to understand the use cases for such an option - which scenarios can benefit from having the ability to save everything aside from the panel editor details?
>
> The whole story around how we persist 'state' between JMRI sessions seems to regularly generate questions and confusion within the user community.
>
> Thoughts?
>
> Best regards,
>
> Matt H


Bob Jacobsen
rgj1927@...








Bob Jacobsen
 

Sorry, wasn’t clear. I really meant “every”

Bob

On Aug 28, 2020, at 11:57 AM, whmvd <@WouterVanDoorn> wrote:

"We could also consider a “store a file on every quit” i.e. YourLastSession.xml”, maybe as an option.

And perhaps simpler ways of setting up the usual “load at startup” paradigm."

I learnt today that closing a session does not necessarily generate the 'do you want to save your changes?' pop-up. The generation of a YourLastSession.xml would, I take it, be an extra action wherever that pop-up is generated, and that would make it similarly unreliable. Otherwise, I'd be all for it. Especially as then you wouldn't need to do anything to the "load at startup", because loading it at start-up with "YourLastSession.xml" would then allow the vast majority of people to forget about the whole store/load business. Sounds pretty ideal, in fact!

Wouter


On Fri, 28 Aug 2020 at 16:52, Bob Jacobsen <@BobJacobsen> wrote:
Historically, it was introduced as a migration tool. I’m note sure it has any use now that we have better tools for moving user names, etc. Maybe leave it in the Debug menu?

We could also consider a “store a file on every quit” i.e. YourLastSession.xml”, maybe as an option.

And perhaps simpler ways of setting up the usual “load at startup” paradigm.

Bob

On Aug 28, 2020, at 11:03 AM, Matthew Harris <@mattharris> wrote:

Dear all,

Given the myriad of confusion surrounding the whole panel storage options - see recent jmriusers thread https://groups.io/g/jmriusers/topic/76465874 as an example of this - is it time for us to consider removing the 'Store Configuration Only' option?

I'm just trying to understand the use cases for such an option - which scenarios can benefit from having the ability to save everything aside from the panel editor details?

The whole story around how we persist 'state' between JMRI sessions seems to regularly generate questions and confusion within the user community.

Thoughts?

Best regards,

Matt H

Bob Jacobsen
@BobJacobsen








Bob Jacobsen
@BobJacobsen


Ken Cameron
 

Bob,

On your last comment 'simpler ways of setting up the usual "load at startup"
paradigm', could it trigger off a 'store as' concept? If you are doing the
store and changing the name from the one that matched a file that is in the
startup action to load, could/should it offer to also update the startup
action? That might make it simpler for users and reduce the 'forgot to load
newly stored file' issue.

At first, I was thinking of some fuzzy matching of the name to detect you
are versioning the name slightly. This would happen during the startup by
seeing a newer file with the fuzzy match, offer to load the newer file and
offer to update the startup action. By fuzzy match I was thinking number of
similar characters and a newer timestamp for an xml file other than the
usual ones we know about. But when you are in the store and make a change, I
think that's a clear trigger we could use to know to offer the extra step.

-Ken Cameron, Member JMRI Dev Team
www.jmri.org
www.fingerlakeslivesteamers.org
www.cnymod.org
www.syracusemodelrr.org


Dave Sand
 

Wouter,

The "remember to store" pop-ups are related to changes to a specific table.  There are no reminders for panel changes.

The problem with automatic store is what happens if you did not like the latest changes and wanted to quit without saving?

Dave Sand

----- Original message -----
From: whmvd <vandoornw@...>
Subject: Re: [jmri-developers] Time to retire 'Store Configuration Only to File'?
Date: Friday, August 28, 2020 10:57 AM

"We could also consider a “store a file on every quit” i.e. YourLastSession.xml”, maybe as an option.

And perhaps simpler ways of setting up the usual “load at startup” paradigm."

I learnt today that closing a session does not necessarily generate the 'do you want to save your changes?' pop-up. The generation of a YourLastSession.xml would, I take it, be an extra action wherever that pop-up is generated, and that would make it similarly unreliable. Otherwise, I'd be all for it. Especially as then you wouldn't need to do anything to the "load at startup", because loading it at start-up with "YourLastSession.xml" would then allow the vast majority of people to forget about the whole store/load business. Sounds pretty ideal, in fact!

Wouter


On Fri, 28 Aug 2020 at 16:52, Bob Jacobsen <rgj1927@...> wrote:
Historically, it was introduced as a migration tool.  I’m note sure it has any use now that we have better tools for moving user names, etc.  Maybe leave it in the Debug menu?

We could also consider a “store a file on every quit” i.e. YourLastSession.xml”, maybe as an option.

And perhaps simpler ways of setting up the usual “load at startup” paradigm.

Bob

> On Aug 28, 2020, at 11:03 AM, Matthew Harris <matthew.john.harris@...> wrote:
>
> Dear all,
>
> Given the myriad of confusion surrounding the whole panel storage options - see recent jmriusers thread https://groups.io/g/jmriusers/topic/76465874 as an example of this - is it time for us to consider removing the 'Store Configuration Only' option?
>
> I'm just trying to understand the use cases for such an option - which scenarios can benefit from having the ability to save everything aside from the panel editor details?
>
> The whole story around how we persist 'state' between JMRI sessions seems to regularly generate questions and confusion within the user community.
>
> Thoughts?
>
> Best regards,
>
> Matt H


Bob Jacobsen









Bob Jacobsen
 

I could certainly see advanced users really hating a “store at the end of every run, bring it back exactly like that”. They probably do lots of interesting/experiemental things, and always having to unwind those instead of just restarting-without-store would be a nightmare.

On the other hand, it’s confusing pain to get the usual “manually store, set preferences to load that file at startup”; there’s confusion about file, etc.

So maybe more than one solution is needed here.

Bob


Bob Jacobsen
@BobJacobsen


danielb987
 

A suggestion, if somebody is interested in coding it:

Have a preference setting there the user can choose between:

* Never store on close
* Always store on close
* Ask the user on store

The last option would give the user a dialog box where the user would select if to store or not. An alternative to a dialog box is two menu items "Close and store" and "Close without store".

Daniel

2020-08-28 18:30 skrev Bob Jacobsen:

I could certainly see advanced users really hating a “store at the end
of every run, bring it back exactly like that”. They probably do lots
of interesting/experiemental things, and always having to unwind those
instead of just restarting-without-store would be a nightmare.
On the other hand, it’s confusing pain to get the usual “manually
store, set preferences to load that file at startup”; there’s
confusion about file, etc.
So maybe more than one solution is needed here.
Bob

Bob Jacobsen
@BobJacobsen


danielb987
 

Sorry, it should be:

Have a preference setting there the user can choose between:

* Never store on close
* Always store on close
* Ask the user on close

Daniel

2020-08-28 18:47 skrev danielb987:

A suggestion, if somebody is interested in coding it:
Have a preference setting there the user can choose between:
* Never store on close
* Always store on close
* Ask the user on store
The last option would give the user a dialog box where the user would
select if to store or not. An alternative to a dialog box is two menu
items "Close and store" and "Close without store".
Daniel
2020-08-28 18:30 skrev Bob Jacobsen:
I could certainly see advanced users really hating a “store at the end
of every run, bring it back exactly like that”. They probably do lots
of interesting/experiemental things, and always having to unwind those
instead of just restarting-without-store would be a nightmare.
On the other hand, it’s confusing pain to get the usual “manually
store, set preferences to load that file at startup”; there’s
confusion about file, etc.
So maybe more than one solution is needed here.
Bob

Bob Jacobsen
@BobJacobsen


Balazs Racz
 

For quite a long while now (over a decade I think) the overwhelming majority of the user interfaces I use on a daily basis do not have a concept of "save". Everything I do is automatically saved. I never lose any data or work, and this gives me a good feeling.
The majority of the computing devices today are using some touch based operating system (phones, tablets), and the clear user expectation there is that you can switch to another application, reboot the device, come back to your app, and you will find almost everything exactly the way you left it.
  
JMRI has almost all the code to do this, but it is not arranged into the right concepts. It is only a question of this arrangement whether our behavior mimics the modern expectation, or the nineties' expectations. This is not about making advanced users angry (every change will make advanced users angry), but labelling the code with the right concepts so that both advanced users know what they need to do and users with modern expectations do not get tripped up by the behavior.

Here is an example of re-labeling the concepts:
- every profile directory has a fixed config XML where all the configuration and panels are stored. Not just at every exit, but also pretty much after every change. This is loaded upon every startup of the given profile. The "load panel at startup" option goes away.
- a new option "Backup configuration and panels" runs the current save flow and allows exporting the objects into an XML file of the user's choosing.
- a mirror option "Restore configuration and panels" overwrites the current configuration with one from an XML file. This may have to restart PanelPro, but that's probably fine, since this is an untypical user journey.
- The "Load panels..." is renamed to "Merge configuration and panels from...". It becomes a very rarely used feature.
- Another rarely used feature would be to "Clear all configuration". This would be equivalent to restoring from an empty XML file (or creating a new profile).
- A currently rarely used feature might become more prominent, to duplicate a profile into a new profile. This would allow safe experimentation with settings that the user may want to discard.

Auto-save however will cause a currently missing feature to be rather painful, and this mandates some work. A very robust "undo" feature would be needed for the user to be able to review past changes and go back if they realize they made a mistake. This is to some extent part of the user contract of "we will never lose the data you entered". A first cut approximation of this could be automatic backups like the operations tool is doing, but having the real deal would be sweet.

Is this too crazy? The operations tool is actually working to a certain extent according to these concepts, for example wrt the car database.

thanks
Balazs




On Fri, Aug 28, 2020 at 6:49 PM danielb987 <db123@...> wrote:
Sorry, it should be:

Have a preference setting there the user can choose between:

* Never store on close
* Always store on close
* Ask the user on close

Daniel

2020-08-28 18:47 skrev danielb987:
> A suggestion, if somebody is interested in coding it:
>
> Have a preference setting there the user can choose between:
>
> * Never store on close
> * Always store on close
> * Ask the user on store
>
> The last option would give the user a dialog box where the user would
> select if to store or not. An alternative to a dialog box is two menu
> items "Close and store" and "Close without store".
>
> Daniel
>
> 2020-08-28 18:30 skrev Bob Jacobsen:
>> I could certainly see advanced users really hating a “store at the end
>> of every run, bring it back exactly like that”. They probably do lots
>> of interesting/experiemental things, and always having to unwind those
>> instead of just restarting-without-store would be a nightmare.
>>
>> On the other hand, it’s confusing pain to get the usual “manually
>> store, set preferences to load that file at startup”; there’s
>> confusion about file, etc.
>>
>> So maybe more than one solution is needed here.
>>
>> Bob
>>
>> —
>> Bob Jacobsen
>> rgj1927@...
>>
>>
>>
>>
>>
>>
>
>




JerryG
 

Daniel’s suggestion is one possibility, especially with default being “ask user” (and with a note in the dialog box that this can be changed in Preferences).  However, I don’t think I’ve ever seen an application with “never store on close,” with most having the approach of “ask user on close if anything changed.”  I understand the concern that sometimes things are changed but that change is somehow not noticed by JMRI before it closes [or, worse, you get a message that says “something changed and you should have stored it” then it closes anyway!]  Is this really off the table for a way to go?  I suspect typical users would really appreciate it.

On the issue of the original post, “Store Configuration Only,” that could still be an additional option when asking the user.  I have had a couple of situations where I wanted all my sensors, TOs, memories, etc. but wanted to develop some alternative panels.  Of course, I could also do this by editing the XML file, but that’s a questionable activity for most, including me!

Jerry


Egbert Broerse
 

I would support Jerry’s suggestion to include “never store on close” in such a change as it would provide sort of a read-only configuration.

Balazs’ idea to always store on macOS at least assumes a way to step back to a previous “version”. It happens to be integrated/look like TimeMachine allowing one to compare a certain content. Could perhaps be presented as a list of back-ups from the Preference > Backup Panels subfolder.

Egbert

On 29 aug. 2020,at 01:31 JerryG via groups.io <jerryg2003=aol.com@groups.io> wrote:

Daniel’s suggestion is one possibility, especially with default being “ask user” (and with a note in the dialog box that this can be changed in Preferences). However, I don’t think I’ve ever seen an application with “never store on close,” with most having the approach of “ask user on close if anything changed.” I understand the concern that sometimes things are changed but that change is somehow not noticed by JMRI before it closes [or, worse, you get a message that says “something changed and you should have stored it” then it closes anyway!] Is this really off the table for a way to go? I suspect typical users would really appreciate it.

[…]

Jerry


Dave Sand
 

The backupPanels directory only contains previous versions when the replacing an existing file. For people who create a new name for each store, their backup files are based on their naming convention.

Dave Sand

----- Original message -----
From: Egbert Broerse <@silverailscolo>
To: jmri@jmri-developers.groups.io
Subject: [jmri-developers] Time to retire 'Store Configuration Only to File'?
Date: Saturday, August 29, 2020 9:44 AM

I would support Jerry’s suggestion to include “never store on close” in such a change as it would provide sort of a read-only configuration.

Balazs’ idea to always store on macOS at least assumes a way to step back to a previous “version”. It happens to be integrated/look like TimeMachine allowing one to compare a certain content. Could perhaps be presented as a list of back-ups from the Preference > Backup Panels subfolder.

Egbert

On 29 aug. 2020,at 01:31 JerryG via groups.io <jerryg2003=aol.com@groups.io> wrote:

Daniel’s suggestion is one possibility, especially with default being “ask user” (and with a note in the dialog box that this can be changed in Preferences). However, I don’t think I’ve ever seen an application with “never store on close,” with most having the approach of “ask user on close if anything changed.” I understand the concern that sometimes things are changed but that change is somehow not noticed by JMRI before it closes [or, worse, you get a message that says “something changed and you should have stored it” then it closes anyway!] Is this really off the table for a way to go? I suspect typical users would really appreciate it.

[…]

Jerry


Bob Jacobsen
 

The store operation was originally quite slow. Disks were slow, XML processing took time, etc. It was thought that _always_ doing it would result in people deciding that the program had hung on exist and killing it; reliably preventing that from corrupting files is hard.

It might or might not be considered slow now; people run JMRI on some pretty low-end hardware. And it might be possible to make the store more robust against failures (though Windows support for various operations differs from macOS/Linux). So perhaps this is worth investigating for the original purpose of this long and winding thread, which was roughly “I quit and lost some work, can I get it back?”.

It’s hard to see “store every time, and use that at the next start up” every time as being a good thing. Yes, there are (non-control system) programs that do this, but they tend to have undo operations, and they don’t tend to be able to drop expensive equipment on the floor. All three of the process control systems I work with have auto-store-for-backup, but those backups are not automatically used after restart; the previously-blessed configs are used after restart (well, EPICS doesn’t really have our concept of “restart”, but there is an analog)

As to having a _reliable_ way to ask to store at close if-and-only-iff something changed: Good luck with that. I’m not sure that’s even theoretically possible short of storing the configuration to a temporary and doing a comparison. Our code is certainly very far from it now.

Can we find a simpler approach for users that’s based on what the code can do now?

Bob

Bob Jacobsen
@BobJacobsen


whmvd
 

A slightly different way of looking at it would be:
- always store at exit, but: to a new name and in a location the user doesn't see as his own
- on every start-up, if a panel is auto-opened that is the same as the one last auto-saved (that's the tricky bit) do a difference of that auto-saved file with the one being opened and ask if those last changes should be retained.

It would need looking at very carefully to keep behaviour (user-)predictable and consistent when load and store panels are being used in a session. Timestamps will certainly come into play, as will exit-processing on load. I also think there'd need to be a sturdy safeguard against accidental merges via load panel on top of existing config.Tough merges should remain possible.

It was a minefield before, it'll be a minefield whichever route is chosen. If the whole load/store processing was easy, it'd have been definitively sorted log ago.

Wouter


On Sat, 29 Aug 2020 at 20:39, Bob Jacobsen <rgj1927@...> wrote:
The store operation was originally quite slow.  Disks were slow, XML processing took time, etc.  It was thought that _always_ doing it would result in people deciding that the program had hung on exist and killing it; reliably preventing that from corrupting files is hard. 

It might or might not be considered slow now; people run JMRI on some pretty low-end hardware.  And it might be possible to make the store more robust against failures (though Windows support for various operations differs from macOS/Linux). So perhaps this is worth investigating for the original purpose of this long and winding thread, which was roughly “I quit and lost some work, can I get it back?”.

It’s hard to see “store every time, and use that at the next start up” every time as being a good thing.  Yes, there are (non-control system) programs that do this, but they tend to have undo operations, and they don’t tend to be able to drop expensive equipment on the floor.  All three of the process control systems I work with have auto-store-for-backup, but those backups are not automatically used after restart; the previously-blessed configs are used after restart (well, EPICS doesn’t really have our concept of “restart”, but there is an analog)

As to having a _reliable_ way to ask to store at close if-and-only-iff something changed:  Good luck with that.  I’m not sure that’s even theoretically possible short of storing the configuration to a temporary and doing a comparison.  Our code is certainly very far from it now.

Can we find a simpler approach for users that’s based on what the code can do now?

Bob

Bob Jacobsen
rgj1927@...








Steve_G
 

There are some things we might consider.
Top left main panel pro screen has a menu item "File", which doesnt have open, close, save, save as, open recent, etc, those are the words 90% of apps I have put there. This is, in my opinion, the biggest problem.
The preferences for never save, prompt, always save. 
The preference for restore states on load, or not.
Under Panels, delete panel, new panel, show panel, list panels.
When opening a layout, show which panels it contains before opening.

I can't think of an app that has an undo after you save and exit, but maybe I haven't looked.

I find the saving conversation interesting as at home and club our biggest concern is how to stop them saving.... All our layouts are set as read only, 

I would like the ability to not auto discover turnouts as kids, old people and drunks can add hundreds of turnouts in a day!!!

Steve G.