Topics

Deprecation Holiday? (was Re: Deprecating jmri.** code in favor of apps.** code


Bob Jacobsen
 

Randall asked for help picking between two unfortunate options forced by our deprecation policy (see below). That policy has added a _bunch_ of extra work to my effort to better organize LayoutEditor, and has delayed some efficiency improvements for almost two years (though that’s timer has now run after 4.20 comes out)

There’s a lot of this stacked up right now, so I’d like to propose declaring this Fall a “deprecation holiday”. Specifically:

1) Changes this Fall _do_ _not_ have to follow our “deprecate changes to APIs” policy. So long as they pass CI and are in the release notes, the JMRI APIs (Classes and methods and parameters) can be changed directly.

2) We call the Fall release JMRI 5 to make sure people know they might have some migratin' to do. (I don’t now whether that’s 4.21.* -> 5.0 or some other sequence to be decided)
2A) In the past, we’ve only changed 1st digit for a Java requirement change. That might or might not happen this Fall, but it’s not what’s being discussed on this thread.

3) We aim a bit earlier for the release, maybe 1st of December, so that we can get a correction release out (if needed, but it will be) before the holiday break. Lots of people are working on trains over that break and update to the production release.

4) After JMRI 5 is out, we’re back to the policy of deprecate-and-wait for significant API changes for a couple years.

Thoughts?

Bob

On Jun 18, 2020, at 4:12 AM, Randall Wood via groups.io <rhwood=mac.com@groups.io> wrote:

I am trying to remove a dependency on Log4J in the jmri.** packages (this is specifically so that jmri.** packages can be used in an application that does not use Log4J for logging).

I have found that this means either removing code from jmri.** without deprecation or making a deprecated class in jmri.** depend on a class in apps.**

How should this be handled?

This question is really effecting jmri.jmrit.DebugMenu (used only by two classes in apps.*), and two entries in that menu; one of which effectively duplicates the SystemConsole, and one of which displays the Log4J configuration.

Randall
--
Bob Jacobsen
@BobJacobsen


danielb987
 

Sounds good. If we do that, PR #8707 would benefit from that. I had to add a temporary interface CommonLight to handle deprecation. With a deprecation holliday, that would not be needed.

#8707: Split Light into Light and Variablelight
https://github.com/JMRI/JMRI/pull/8707

Also, there is four methods in jmri.Manager that could be removed. If they are removed, two deprecated interfaces from Manager could also be removed.

public List<String> getSystemNameList();
public List<E> getNamedBeanList();
public void addDataListener(ManagerDataListener<E> e);
public void removeDataListener(ManagerDataListener<E> e);
public default void setDataListenerMute(boolean muted);

interface ManagerDataListener<E extends NamedBean>;
public final class ManagerDataEvent<E extends NamedBean> extends java.util.EventObject;

Daniel

2020-06-18 18:41 skrev Bob Jacobsen:

Randall asked for help picking between two unfortunate options forced
by our deprecation policy (see below). That policy has added a
_bunch_ of extra work to my effort to better organize LayoutEditor,
and has delayed some efficiency improvements for almost two years
(though that’s timer has now run after 4.20 comes out)
There’s a lot of this stacked up right now, so I’d like to propose
declaring this Fall a “deprecation holiday”. Specifically:
1) Changes this Fall _do_ _not_ have to follow our “deprecate changes
to APIs” policy. So long as they pass CI and are in the release
notes, the JMRI APIs (Classes and methods and parameters) can be
changed directly.
2) We call the Fall release JMRI 5 to make sure people know they might
have some migratin' to do. (I don’t now whether that’s 4.21.* -> 5.0
or some other sequence to be decided)
2A) In the past, we’ve only changed 1st digit for a Java requirement
change. That might or might not happen this Fall, but it’s not what’s
being discussed on this thread.
3) We aim a bit earlier for the release, maybe 1st of December, so
that we can get a correction release out (if needed, but it will be)
before the holiday break. Lots of people are working on trains over
that break and update to the production release.
4) After JMRI 5 is out, we’re back to the policy of deprecate-and-wait
for significant API changes for a couple years.
Thoughts?
Bob

On Jun 18, 2020, at 4:12 AM, Randall Wood via groups.io <rhwood=mac.com@groups.io> wrote:
I am trying to remove a dependency on Log4J in the jmri.** packages (this is specifically so that jmri.** packages can be used in an application that does not use Log4J for logging).
I have found that this means either removing code from jmri.** without deprecation or making a deprecated class in jmri.** depend on a class in apps.**
How should this be handled?
This question is really effecting jmri.jmrit.DebugMenu (used only by two classes in apps.*), and two entries in that menu; one of which effectively duplicates the SystemConsole, and one of which displays the Log4J configuration.
Randall
--
Bob Jacobsen
@BobJacobsen


Bob Jacobsen
 

The DataListener stuff is the performance improvement I mentioned. It was deprecated by mistake (see comment in #8534) so that will be reversed shortly. While Managers needed to maintain both sets and lists (the other deprecated methods) it was really hard to complete that, but once the lists are gone it’ll significantly improve the GUI speed of large tables. (There’s some work that needs to be done to remove the last 100 or so uses of getSystemNameList too)

Bob

On Jun 18, 2020, at 10:07 AM, danielb987 <db123@...> wrote:

Sounds good. If we do that, PR #8707 would benefit from that. I had to add a temporary interface CommonLight to handle deprecation. With a deprecation holliday, that would not be needed.

#8707: Split Light into Light and Variablelight
https://github.com/JMRI/JMRI/pull/8707

Also, there is four methods in jmri.Manager that could be removed. If they are removed, two deprecated interfaces from Manager could also be removed.

public List<String> getSystemNameList();
public List<E> getNamedBeanList();
public void addDataListener(ManagerDataListener<E> e);
public void removeDataListener(ManagerDataListener<E> e);
public default void setDataListenerMute(boolean muted);

interface ManagerDataListener<E extends NamedBean>;
public final class ManagerDataEvent<E extends NamedBean> extends java.util.EventObject;

Daniel

2020-06-18 18:41 skrev Bob Jacobsen:
Randall asked for help picking between two unfortunate options forced
by our deprecation policy (see below). That policy has added a
_bunch_ of extra work to my effort to better organize LayoutEditor,
and has delayed some efficiency improvements for almost two years
(though that’s timer has now run after 4.20 comes out)
There’s a lot of this stacked up right now, so I’d like to propose
declaring this Fall a “deprecation holiday”. Specifically:
1) Changes this Fall _do_ _not_ have to follow our “deprecate changes
to APIs” policy. So long as they pass CI and are in the release
notes, the JMRI APIs (Classes and methods and parameters) can be
changed directly.
2) We call the Fall release JMRI 5 to make sure people know they might
have some migratin' to do. (I don’t now whether that’s 4.21.* -> 5.0
or some other sequence to be decided)
2A) In the past, we’ve only changed 1st digit for a Java requirement
change. That might or might not happen this Fall, but it’s not what’s
being discussed on this thread.
3) We aim a bit earlier for the release, maybe 1st of December, so
that we can get a correction release out (if needed, but it will be)
before the holiday break. Lots of people are working on trains over
that break and update to the production release.
4) After JMRI 5 is out, we’re back to the policy of deprecate-and-wait
for significant API changes for a couple years.
Thoughts?
Bob
On Jun 18, 2020, at 4:12 AM, Randall Wood via groups.io <rhwood=mac.com@groups.io> wrote:
I am trying to remove a dependency on Log4J in the jmri.** packages (this is specifically so that jmri.** packages can be used in an application that does not use Log4J for logging).
I have found that this means either removing code from jmri.** without deprecation or making a deprecated class in jmri.** depend on a class in apps.**
How should this be handled?
This question is really effecting jmri.jmrit.DebugMenu (used only by two classes in apps.*), and two entries in that menu; one of which effectively duplicates the SystemConsole, and one of which displays the Log4J configuration.
Randall
--
Bob Jacobsen
@BobJacobsen

--
Bob Jacobsen
@BobJacobsen


Bob M.
 

Bob J. and all,

I support the idea, and believe that it to be in the best interest of our customers.

But we should go into this with the full understanding that there will be a set of JMRI users who will _not_ migrate their stuff but _expect_ that their old JMRI install will be kept "up-to-date" with respect to the mainstream releases.

If we indeed go this route, we _need to set the expectation_ that the old releases and mechanisms will be "cut off" from support altogether, and that migration is _required_ for support of new JMRI functionality.

And it may be helpful to start the "marketing" sooner rather than later. Else the JMRIusers list will be filled both with "I don't know how to migrate" issues in addition to the "I just started using JMRI" issues and "why can't my hardware read decoder XXX" issues and "when will JMRI support decoder YYY" issues, all at end-of-year holiday time when we are all pulled in so many different directions!

(And I wonder if the deprecation window should be so long. We make 3 or 4 "major" releases in a 2 year window, with signficant and sometimes radical changes over that time. Anyone who decides to stay with an old release will likley stay there until he is dragged, kicking and screaming, to a more-modern release, because of Java version support sunset, O/S support changes, or simply wanting to be able to use the symbolic programmer with that new loco he just got. Any JMRI user who cares about new JMRI functionality will make the move to migrate to new features, whatever the cost. I feel that "coddling" those who decide to stay with an old release for _two years_ is excessive, especially given the amount of support they generally require once they try to make the move.)

Regards,
Bob M.


Paul Bender
 

On Jun 18, 2020, at 4:40 PM, Bob M. <jawhugrps@...> wrote:
But we should go into this with the full understanding that there will be a set of JMRI users who will _not_ migrate their stuff but _expect_ that their old JMRI install will be kept "up-to-date" with respect to the mainstream releases.
If we indeed go this route, we _need to set the expectation_ that the old releases and mechanisms will be "cut off" from support altogether, and that migration is _required_ for support of new JMRI functionality.
The expectations we have always set for users is that we don’t go back and add new features to old releases. In fact we go further, and say that once you upgrade, we don’t guarantee you can go back to an older release.

But the depreciation cycle is not about user level features. When we deprecate code that is for the Rodney Black’s of the world who are writing software based on JMRI. ( Rodney is the author of CATS ). We are giving other developers, who use our code as a library, a notice that the API is changing. Sometimes this coincides with am implementation behavior change, but not always.

We keep more typical users happy by having a set of tests that protect the external behavior of features they use. Relating this back to the work that Bob is doing with the layout editor... users don’t care what the internal mechanisms are that make their panel work. Users care that their panel loads and displays correctly and that any behavior they have configured ( logix, routes, etc ) behaves as it did when they configured it.

So when we move to 5.0, the Deprecation Holiday should not give us license to break ANY file that loads and does something useful in 4.20. It does give us license to change the structure and/or organization of the code without having to worry about providing a transitional code path from the old structure to the new one. The version number jump in and of itself should be a clue to outside developers that major changes may have occurred.

Paul


Bob Jacobsen
 

Well said.

Bob

On Jun 18, 2020, at 5:17 PM, Paul Bender <paul.bender@...> wrote:

But the depreciation cycle is not about user level features. When we deprecate code that is for the Rodney Black’s of the world who are writing software based on JMRI. ( Rodney is the author of CATS ). We are giving other developers, who use our code as a library, a notice that the API is changing. Sometimes this coincides with am implementation behavior change, but not always.

We keep more typical users happy by having a set of tests that protect the external behavior of features they use. Relating this back to the work that Bob is doing with the layout editor... users don’t care what the internal mechanisms are that make their panel work. Users care that their panel loads and displays correctly and that any behavior they have configured ( logix, routes, etc ) behaves as it did when they configured it.

So when we move to 5.0, the Deprecation Holiday should not give us license to break ANY file that loads and does something useful in 4.20. It does give us license to change the structure and/or organization of the code without having to worry about providing a transitional code path from the old structure to the new one. The version number jump in and of itself should be a clue to outside developers that major changes may have occurred.
--
Bob Jacobsen
@BobJacobsen


Bob M.
 

I seem to remember past issues on the JMRI list where user-written scripts (and even distributed scripts in the jython directory!) failed after upgrade to a JMRI version where deprecated code had been removed. That is the source of those comments.

But I guess I must be mistaken.

Regards,
Bob M.


Ken Cameron
 

Bob M,

You are right that missing some of the changes the scripts need is a
problem. Usually it is customized scripts of a user that started with the
production scripts. Other times we've missed making the updates in the
production scripts for some depreciations. That later case we should be able
to prevent. The former is harder. Best I've seen is a set of examples of
what we had to change in the production scripts so they can match the
failing lines and make the same updated we did in production. But catching
their eye that these things need to be done, that has been the hardest issue
over time. Hopefully, the sequence number change will make them think about
the scope of the changes and that any private script will likely need
inspection for needing updates.

-Ken Cameron, Member JMRI Dev Team
www.jmri.org
www.fingerlakeslivesteamers.org
www.cnymod.org
www.syracusemodelrr.org


Ken Cameron
 

I think there is reasonable cause for doing this. Only question would be if
there were some simple things that are currently being held for 4.21.1 that
might be better pulled into 4.20. I ask this as in the past it was common
that somethings were a little late for a cutoff and a number of users were
waiting on that X.Y.1 release for little things.

Then the new series would jump to 4.99.1 to show how it is the 'new
territory' as we converge to the 5.0 version towards year end. If that
number change is possible, it would help them know it has some things that
are a bit of jump from 4.20 and not just some simple incremental stuff.

-Ken Cameron, Member JMRI Dev Team
www.jmri.org
www.fingerlakeslivesteamers.org
www.cnymod.org
www.syracusemodelrr.org


Bob Jacobsen
 

To keep this moving toward a concrete decision, here’s what I’d put in the release notes in the “warnings” part:

The JMRI (need to decide on name for the series) development sequence will be changing a number of the internal interfaces. This means:
- Panel files, profiles and other files created or updated by JMRI (series name) or later will likely not be readable by JMRI 4.20 or earlier versions
- If you are calling into the JMRI code from custom scripts or other kinds of custom code, you likely have to adjust them to use them with JMRI (series) and later releases.
And in the "Update From Older JMRI Versions” part, add one at the top:

If you are currently using JMRI 4.19.8 or earlier, we strongly recommend that you update to
4.20 and make sure that's running without errors during startup or on the JMRI log.
Once you have JMRI 4.20 running OK, store your configuration and panel files and use those from then on.
There have been changes since since earlier releases,
and moving to the stable <a href="jmri4.12.shtml">JMRI 4.12</a> release
is a good way to work through any possible problems.

and for each change, mention the specific classes that were changed in the release note:

The VampireSorter class has been changed: the ‘doAccent(..)’ call has been changed, please use ’setSortOnAccent(..)'
In addition, mention that this is a migratory release series briefly announcement emails, linked pages, etc.

The JMRI () series will have internal changes that might affect custom scripts or other software that uses JMRI internals. See the release notes for more information.

Any recommendations on how to make it better?

Any other places to put comments,?

If we decide to do this, do we call the next development series 4.21.*, 4.99.*, 5.0.* or something else? (If we want to really change to semantic versioning, this is _really_ the time)

Thanks for the consideration

Bob

--
Bob Jacobsen
@BobJacobsen


danielb987
 

One thought is that since JMRI 4.20 hasn't come out yet, you could put a note in there that the next version will have major changes. I'm thinking particular on CATS and similar projects that that way would get a heads up that something is happening in the next version.

I vote for calling the next production release JMRI 5.0. So for the next development series, 4.99.* seems good.

Daniel


Dave Sand
 

While this thread is about deprecating code, do there need to be rules about data? The main PanelPro xml file does have a schema reference, although that was not changed for a long time even with changes that were not backward compatible.

The sample "warnings" included a section on panels and other data. For example, 4.19.5, or earlier, cannot load a layout editor panel created or updated by 4.19.6 or later.

Dave Sand

----- Original message -----
From: Bob Jacobsen <@BobJacobsen>
To: jmri@jmri-developers.groups.io
Subject: Re: [jmri-developers] Deprecation Holiday? (was Re: Deprecating jmri.** code in favor of apps.** code
Date: Friday, June 19, 2020 1:11 PM

To keep this moving toward a concrete decision, here’s what I’d put in the release notes in the “warnings” part:

The JMRI (need to decide on name for the series) development sequence will be changing a number of the internal interfaces. This means:
- Panel files, profiles and other files created or updated by JMRI (series name) or later will likely not be readable by JMRI 4.20 or earlier versions
- If you are calling into the JMRI code from custom scripts or other kinds of custom code, you likely have to adjust them to use them with JMRI (series) and later releases.
And in the "Update From Older JMRI Versions” part, add one at the top:

If you are currently using JMRI 4.19.8 or earlier, we strongly recommend that you update to
4.20 and make sure that's running without errors during startup or on the JMRI log.
Once you have JMRI 4.20 running OK, store your configuration and panel files and use those from then on.
There have been changes since since earlier releases,
and moving to the stable <a href="jmri4.12.shtml">JMRI 4.12</a> release
is a good way to work through any possible problems.

and for each change, mention the specific classes that were changed in the release note:

The VampireSorter class has been changed: the ‘doAccent(..)’ call has been changed, please use ’setSortOnAccent(..)'
In addition, mention that this is a migratory release series briefly announcement emails, linked pages, etc.

The JMRI () series will have internal changes that might affect custom scripts or other software that uses JMRI internals. See the release notes for more information.

Any recommendations on how to make it better?

Any other places to put comments,?

If we decide to do this, do we call the next development series 4.21.*, 4.99.*, 5.0.* or something else? (If we want to really change to semantic versioning, this is _really_ the time)

Thanks for the consideration

Bob

--
Bob Jacobsen
@BobJacobsen


whmvd
 

Hi Bob,

This bit:
  > If you are currently using JMRI 4.19.8 or earlier, we strongly recommend that you update to
   > 4.20 and make sure that's running without errors during startup or on the JMRI log. 
   > Once you have JMRI 4.20 running OK, store your configuration and panel files and use those from then on.
   > There have been changes since since earlier releases,
   > and moving to the stable <a href="jmri4.12.shtml">JMRI 4.12</a> release
   > is a good way to work through any possible problems.
seems a bit illogically ordered, as it describes first what should come after what it describes second. I'd prefer it like this:

> If you are currently using a version older than 4.20, please follow these instructions carefully:
> 1) Is your current version older than 4.12? Then update to 4.12 first, and ensure that works correctly (check
>     start-up, check things you need to be working, and check the JMRI log as well) before doing anything else
>     Store your configuration and panel files under a new name and use those from now on.
>     Continue with step 2.
> 2) Is your current version older than 4.20? Then update to 4.20 first, and ensure that works correctly (check
>     start-up, check things you need to be working, and check the JMRI log as well) before doing anything else
>     Store your configuration and panel files under a new name and use those from now on.
> Following this order of updating via stable major releases safeguards you against possible issues that may arise
> from skipping these versions.

As to the development version to use next, I'd simply stick with 4.21.*, because that leaves 5.0 as the next (recognisably) big one, without
making it impossible to put in a 4.22 should the need arise (however unlikely).

Wouter

On Fri, 19 Jun 2020 at 19:11, Bob Jacobsen <rgj1927@...> wrote:
To keep this moving toward a concrete decision, here’s what I’d put in the release notes in the “warnings” part:

> The JMRI (need to decide on name for the series) development sequence will be changing a number of the internal interfaces. This means:
>  - Panel files, profiles and other files created or updated by JMRI (series name) or later will likely not be readable by JMRI 4.20 or earlier versions
>  - If you are calling into the JMRI code from custom scripts or other kinds of custom code, you likely have to adjust them to use them with JMRI (series) and later releases.

And in the "Update From Older JMRI Versions” part, add one at the top:

> If you are currently using JMRI 4.19.8 or earlier, we strongly recommend that you update to
> 4.20 and make sure that's running without errors during startup or on the JMRI log. 
> Once you have JMRI 4.20 running OK, store your configuration and panel files and use those from then on.
> There have been changes since since earlier releases,
> and moving to the stable <a href="jmri4.12.shtml">JMRI 4.12</a> release
> is a good way to work through any possible problems.


and for each change, mention the specific classes that were changed in the release note:

> The VampireSorter class has been changed:  the ‘doAccent(..)’ call has been changed, please use ’setSortOnAccent(..)'

In addition, mention that this is a migratory release series briefly announcement emails, linked pages, etc.

> The JMRI () series will have internal changes that might affect custom scripts or other software that uses JMRI internals.  See the release notes for more information.


Any recommendations on how to make it better?

Any other places to put comments,?

If we decide to do this, do we call the next development series 4.21.*, 4.99.*, 5.0.* or something else?  (If we want to really change to semantic versioning, this is _really_ the time)

Thanks for the consideration

Bob

--
Bob Jacobsen
rgj1927@...







Bob Jacobsen
 

The schema reference only has to update when there’s a schema change that older panels can’t pass. For example, adding something optional doesn’t require one, as old files with the old schema can still be read. Adding something mandatory or removing something optional requires changing the schema (so the files pass), but _also_ requires care in the code so that old and new can still be read.

https://www.jmri.org/help/en/html/doc/Technical/XmlSchema.shtml#versioning

I’m hesitant to add specific backwards compatible warnings (4.19.6 cannot be read by 4.19.5 or earlier) because they lead to problems with both false negatives (“Well, my panel worked OK”) and false positives (“well, you didn’t say anything about…”). Perhaps we can add a stronger (stronger words, stronger typography) general statement that if you write files with a newer version you should not expect them to work in older versions, so keep backups when trying a new version?

Bob

On Jun 19, 2020, at 12:07 PM, Dave Sand <@davesand> wrote:

While this thread is about deprecating code, do there need to be rules about data? The main PanelPro xml file does have a schema reference, although that was not changed for a long time even with changes that were not backward compatible.

The sample "warnings" included a section on panels and other data. For example, 4.19.5, or earlier, cannot load a layout editor panel created or updated by 4.19.6 or later.

Dave Sand


----- Original message -----
From: Bob Jacobsen <@BobJacobsen>
To: jmri@jmri-developers.groups.io
Subject: Re: [jmri-developers] Deprecation Holiday? (was Re: Deprecating jmri.** code in favor of apps.** code
Date: Friday, June 19, 2020 1:11 PM

To keep this moving toward a concrete decision, here’s what I’d put in the release notes in the “warnings” part:

The JMRI (need to decide on name for the series) development sequence will be changing a number of the internal interfaces. This means:
- Panel files, profiles and other files created or updated by JMRI (series name) or later will likely not be readable by JMRI 4.20 or earlier versions
- If you are calling into the JMRI code from custom scripts or other kinds of custom code, you likely have to adjust them to use them with JMRI (series) and later releases.
And in the "Update From Older JMRI Versions” part, add one at the top:

If you are currently using JMRI 4.19.8 or earlier, we strongly recommend that you update to
4.20 and make sure that's running without errors during startup or on the JMRI log.
Once you have JMRI 4.20 running OK, store your configuration and panel files and use those from then on.
There have been changes since since earlier releases,
and moving to the stable <a href="jmri4.12.shtml">JMRI 4.12</a> release
is a good way to work through any possible problems.

and for each change, mention the specific classes that were changed in the release note:

The VampireSorter class has been changed: the ‘doAccent(..)’ call has been changed, please use ’setSortOnAccent(..)'
In addition, mention that this is a migratory release series briefly announcement emails, linked pages, etc.

The JMRI () series will have internal changes that might affect custom scripts or other software that uses JMRI internals. See the release notes for more information.

Any recommendations on how to make it better?

Any other places to put comments,?

If we decide to do this, do we call the next development series 4.21.*, 4.99.*, 5.0.* or something else? (If we want to really change to semantic versioning, this is _really_ the time)

Thanks for the consideration

Bob

--
Bob Jacobsen
@BobJacobsen







--
Bob Jacobsen
@BobJacobsen


Randall Wood <rhwood@...>
 

Since I think I triggered all this, here my thoughts on this:

We should not have a "deprecation holiday”, mostly because I don’t want to think of this a temporary break in an existing way of doing things that can be returned to.

That said, I don’t think we should continue the way were going either, for the following reasons:

  • Our version number feel semantic, but are not (see https://semver.org)
  • JMRI as a library is too tightly coupled to the needs of the applications PanelPro and DecoderPro applications, and we should probably break that (I’m not suggesting breaking the repo into multiple repos)
  • We are carrying lots of complexity and some untestable code that could be eliminated by moving our base to the most recent LTS version of Java (11 at the time of writing)
  • We have code in JMRI that is retained for Java 8 compatibility that is incompatible with the announced changes in Java 15

Therefor, I propose the following concerning version names and release cycles:

  • 4.20 be the last 4.x(.0) release

    • 4.20.y releases be made available for either six months or a year for fixes (bug fix and new decoder definitions only)*
  • We cease making monthly test releases with only biannual production releases and switch to a releasing production releases bi-monthly with the only test releases only a couple of weeks ahead of the production release

    • This would avoid the confusion of releasing 4.18 and 4.19.1 at roughly the same time
    • It is clear that some users are actively encouraging other users to not make a distinction between test and production releases and “just use the latest”, while other users feel stuck without support because a production release has a bug that won’t be fixed until the next production release (i.e. some users think we are implying that production still means that fixes will be available for production releases while test releases are expected to be broken in interesting ways)
  • Releases of the JMRI JAR file(s) be versioned strictly using semantic versioning

    • This would mean no removal of deprecated code without a major version change to the JAR file
    • This should make it easier to incorporate JMRI JARs as libraries into other applications by making it clear that, for example, 5.4 has no breaking changes from 5.3 (we don't currently make any such promise for any two versions of JMRI)
    • This means using non-numeric qualifiers to denote pre-releases like SNAPSHOT (a Maven convention), or Alpha and Beta
  • Release of the JMRI application be versioned using calendar versioning (like Eclipse does, as YYYY-MM)

    • Using two different version numbers should not be construed as supporting different release schedules for the libraries and applications, but should be construed as allowing the implication of two different things to two different target audiences
  • The nightlies on Jenkins and personal builds need to clearly be built with version numbers that scream “I’m a nightly build” with all that implies (Its confusing to have users report bugs against 4.21.1 when 4.20 hasn’t been released), i.e. the version needs to be something like "nightly-"

Having proposed that we use the versioning above, I further propose:

  1. We plan what must be in library 5.0 now (its not a calendar event—I know I proposed rapid releases above, but the transition not need be rapidly released)
    • Is it a break from Java 8 to Java 11?
      • Does it come with a policy statement on how we depend on Java? (if so that should be separate discussion)
    • Do we need to break the circular dependencies between apps.** and jmri.**? (I’m a “yes” on all those questions)
  2. When we release 4.20, we announce the change in release cadence (if any), versioning (if any), and what the planned features / breaking changes of the next release will be and how we will handle patching 4.20 if deemed necessary, and for how long we will consider doing that
  3. We consider what changes to our development process may help
    • For example, we could follow the rules of “Conventional Commits” (https://www.conventionalcommits.org/en/v1.0.0/) to allow the change log and version numbers for the JMRI libraries to be automatically generated (I’m not suggesting this at this point)
    • For example, we could split apps and jmri into separate sub-projects (which would force one to be able to compile without the other)

Notes: *: maintaining a master and a 4.20.y development branch will be a pain; I know, but I also think that since we were not in a position when 4.18 was released to state that 4.20 would be the last 4.x release, we need to support it if only for those hobbyists who want to stay current but need to plan computer hardware and OS upgrades to use post-4.20 JMRI. This would be just bug fixes because bugs will be found in 4.20, and new decoder definitions barring those that require feature changes in JMRI would allow the purchase of some current models to continue while planning the jump to post 4.20.


Peter Ulvestad
 

The parts of Randall's post that I understand seems reasonable to me. I like the bi-monthly release proposal with 4.20 being the last 4.x.x release.

--
Peter Ulvestad

JMRI Users Group Moderator - http://www.jmri.org ( http://www.jmri.org )
Tam Valley Group Moderator - https://tamvalleydepot.com/ ( http://tamvalleydepot.com/ )
Sprog-DCC Group Moderator - http://www.sprog-dcc.co.uk/ ( http://www.sprog-dcc.co.uk/ )
Edmonton Model Railroad Association - http://www.emra.club/


Paul Bender
 




On Jun 20, 2020, at 10:01 AM, Randall Wood via groups.io <rhwood@...> wrote:
  • Our version number feel semantic, but are not (see https://semver.org)
  • JMRI as a library is too tightly coupled to the needs of the applications PanelPro and DecoderPro applications, and we should probably break that (I’m not suggesting breaking the repo into multiple repos)

I completely agree with the coupling.  This is particularly clear in that items such as menus are defined in the jmri.jmrit package, when they really should be application details.

  • We are carrying lots of complexity and some untestable code that could be eliminated by moving our base to the most recent LTS version of Java (11 at the time of writing)

While I agree that we have a lot of code that wasn’t written with testing in mind, switching language versions isn’t a panacea for those problems.
  • We have code in JMRI that is retained for Java 8 compatibility that is incompatible with the announced changes in Java 15

Therefor, I propose the following concerning version names and release cycles:

  • 4.20 be the last 4.x(.0) release

    • 4.20.y releases be made available for either six months or a year for fixes (bug fix and new decoder definitions only)*
  • We cease making monthly test releases with only biannual production releases and switch to a releasing production releases bi-monthly with the only test releases only a couple of weeks ahead of the production release


For me to be comfortable with bi-monthly production releases, we need to be much more disciplined about testing.

Tests are our way of checking that changes don’t break the design.  

Also, the build process may need some tweaks so that someone other than bob can perform all the steps.  Asking Bob to do more frequent releases is not something I think we should do.

    • This would avoid the confusion of releasing 4.18 and 4.19.1 at roughly the same time

There is a clear advantage to that.
  • Releases of the JMRI JAR file(s) be versioned strictly using semantic versioning

    • This would mean no removal of deprecated code without a major version change to the JAR file
    • This should make it easier to incorporate JMRI JARs as libraries into other applications by making it clear that, for example, 5.4 has no breaking changes from 5.3 (we don't currently make any such promise for any two versions of JMRI)
    • This means using non-numeric qualifiers to denote pre-releases like SNAPSHOT (a Maven convention), or Alpha and Beta

If we are going to do this, we should start distributing the jar file via Maven Central.  Everything is configured for this to work, and a few of us do have permission to publish the jars.  ( I.e. mvn clean deploy will send the artifacts up where they get staged for release ).

Having proposed that we use the versioning above, I further propose:

  1. We plan what must be in library 5.0 now (its not a calendar event—I know I proposed rapid releases above, but the transition not need be rapidly released)
    • Is it a break from Java 8 to Java 11?
      • Does it come with a policy statement on how we depend on Java? (if so that should be separate discussion)
    • Do we need to break the circular dependencies between apps.** and jmri.**? (I’m a “yes” on all those questions)

The only question I ask here is what advantages are there to moving to Java 11 now.  I am just not well enough versed in changes to language features to be able to comment.

    • For example, we could split apps and jmri into separate sub-projects (which would force one to be able to compile without the other)

I think we are moving in this direction.  That is part of what spurred this discussion.

Paul


Randall Wood <rhwood@...>
 

On MacOS, menus are pure Java when run in a Java 8 JVM, but are written in JavaScript for Nashorn when run in a Java 9-14 JVM (this keeps the handling of the JMRI application menus separate from user scripts). Nashorn has been removed in Java 15. If we required Java 11, the MacOS menus would be pure Java. If do not, we would have to either handle JMRI application menus in the same Jython interpreter that user scripts are running in, or provide special handling to allow GraalVM to be used on Java 15 or newer and maintain three different code bases for three different versions of Java (8, 9-14, 15+).

That is primary reason I have for moving to Java 11—we get one set of code for all JVMs 9+ to handle the menus on MacOS and don’t have to dance around hard compilation and runtime incompatibilities between versions.

Additional reasons that I would like to be able to leverage are (with Java version that introduced feature):
- the “var” keyword for variables in code blocks (10+)
- properties files can be UTF-8 and are not limited to ASCII (9+)
- Set.of(…), Map.of(…), List.of(…) factory methods for collections (9+)
- private and protected methods in Interfaces (9+)
- @Deprecated gains arguments to specify if deprecated means to be removed or merely to be discouraged (9+)
- Possibly the use GraalVM’s multi-language features to provide post-Nashorn JavaScript and Python 3 support and build time support for Node JS (i.e. we would be able to unit test (not just integration test) the Javascript code for web panels) (11+)

On 20-Jun-2020, at 17:18, Paul Bender <paul.bender@...> wrote:




On Jun 20, 2020, at 10:01 AM, Randall Wood via groups.io <rhwood@...> wrote:
  • Our version number feel semantic, but are not (see https://semver.org)
  • JMRI as a library is too tightly coupled to the needs of the applications PanelPro and DecoderPro applications, and we should probably break that (I’m not suggesting breaking the repo into multiple repos)

I completely agree with the coupling.  This is particularly clear in that items such as menus are defined in the jmri.jmrit package, when they really should be application details.

  • We are carrying lots of complexity and some untestable code that could be eliminated by moving our base to the most recent LTS version of Java (11 at the time of writing)

While I agree that we have a lot of code that wasn’t written with testing in mind, switching language versions isn’t a panacea for those problems.
  • We have code in JMRI that is retained for Java 8 compatibility that is incompatible with the announced changes in Java 15

Therefor, I propose the following concerning version names and release cycles:

  • 4.20 be the last 4.x(.0) release

    • 4.20.y releases be made available for either six months or a year for fixes (bug fix and new decoder definitions only)*
  • We cease making monthly test releases with only biannual production releases and switch to a releasing production releases bi-monthly with the only test releases only a couple of weeks ahead of the production release


For me to be comfortable with bi-monthly production releases, we need to be much more disciplined about testing.

Tests are our way of checking that changes don’t break the design.  

Also, the build process may need some tweaks so that someone other than bob can perform all the steps.  Asking Bob to do more frequent releases is not something I think we should do.

    • This would avoid the confusion of releasing 4.18 and 4.19.1 at roughly the same time

There is a clear advantage to that.
  • Releases of the JMRI JAR file(s) be versioned strictly using semantic versioning

    • This would mean no removal of deprecated code without a major version change to the JAR file
    • This should make it easier to incorporate JMRI JARs as libraries into other applications by making it clear that, for example, 5.4 has no breaking changes from 5.3 (we don't currently make any such promise for any two versions of JMRI)
    • This means using non-numeric qualifiers to denote pre-releases like SNAPSHOT (a Maven convention), or Alpha and Beta

If we are going to do this, we should start distributing the jar file via Maven Central.  Everything is configured for this to work, and a few of us do have permission to publish the jars.  ( I.e. mvn clean deploy will send the artifacts up where they get staged for release ).

Having proposed that we use the versioning above, I further propose:

  1. We plan what must be in library 5.0 now (its not a calendar event—I know I proposed rapid releases above, but the transition not need be rapidly released)
    • Is it a break from Java 8 to Java 11?
      • Does it come with a policy statement on how we depend on Java? (if so that should be separate discussion)
    • Do we need to break the circular dependencies between apps.** and jmri.**? (I’m a “yes” on all those questions)

The only question I ask here is what advantages are there to moving to Java 11 now.  I am just not well enough versed in changes to language features to be able to comment.

    • For example, we could split apps and jmri into separate sub-projects (which would force one to be able to compile without the other)

I think we are moving in this direction.  That is part of what spurred this discussion.

Paul


Dave Heap
 

It would be good for us to know now what the foreshadowed requirements are. Some of us are locked into older OS versions due to hardware and/or legacy software constraints. Furthermore some of us are retired pensioners and may not be able to budget for hardware upgrades.

This applies to both developers and users.

At present I feel quite in the dark about things like what Java version we plan to move to and where we can find out about its system requirements.

Dave

On 21 Jun 2020, at 12:01 AM, Randall Wood via groups.io <rhwood=mac.com@groups.io> wrote:

need to plan computer hardware and OS upgrades to use post-4.20 JMRI.


Bob Jacobsen
 

On Jun 20, 2020, at 6:53 PM, Dave Heap <dgheap@...> wrote:

It would be good for us to know now what the foreshadowed requirements are. Some of us are locked into older OS versions due to hardware and/or legacy software constraints. Furthermore some of us are retired pensioners and may not be able to budget for hardware upgrades.

This applies to both developers and users.

At present I feel quite in the dark about things like what Java version we plan to move to and where we can find out about its system requirements.
A long time ago, we started a couple pages on that. I’ve been periodically updating them, but they’re not in great shape:

https://www.jmri.org/help/en/html/doc/Technical/JVMCapabilities.shtml

https://www.jmri.org/help/en/html/doc/Technical/TechRoadMap.shtml

Bob