Topics

Surveying users on use of help/web #WebHelpUpdate


JerryG
 

As some of you know, I’ve been working on reviewing the files in help/en/html and related areas for some time now.  With posts to the user forum approaching 2000/month, I’m wondering whether work on the help/web information is having an impact. I know that many of you reference web pages in your responses, but is there more we can do (example: the PanelPro Getting Started Guide Dave Sands is working on and the Saving Your Work pages I’m doing)?

There are many things still on my task list for reviewing help/web information (see https://github.com/JMRI/JMRI/issues/8139).  What started as a “simple” task to improve help on scripting as I learned it myself (of course with lots of answers from the user and developer community) has blossomed into a major open ended project (as is all of JMRI!).  It would help me (and perhaps any others of you interested in improving help information - hint, hint) to know if/how much this is being used or whether there are areas of particular priority.  I get some feedback from this list, but would like to go to the broader user population for input.

Have any of you done any surveys of the user community?  Any reason NOT to? Groups.io has a polling feature (although not activated for the JMRI users forum, or this forum, I assume it can be by the moderator) so might be relatively easy to get some simple feedback.  

I thought to create a poll that won't take more than 5 minutes for someone to answer.  As you will see, the poll can also be an educational tool for the user community as well.  Some of the questions I’m thinking about:

————
1. When you are working with JMRI, do you generally have access to the internet (on a smartphone, pad, or computer) to get help and other information?
[Note: not necessary, but desirable as access to the user forum requires access to the internet]

2. How often do you [on a scale from "almost every time I use JMRI" to "almost never"]:
 a. Use Help from a JMRI window?
 b. Access the website JMRI.org to get help?
 c. Ask questions in this user forum as my primary source of help?
 d. Use other web resources (blogs, YouTube, etc.) for help?
[Note: all these resources are constantly being updated so check back often if you don’t find what you want the first time.]

3. Do you [scale: “every time I use help resources” to “almost never” and “didn’t know this existed”]:
 a. Find what I need in the help/web system?
 b. Find errors in the help/web information?
 c. Use the JMRI Table of Contents to find things?
 d. Use the JMRI Index to find things?
 h. Use the JMRI Glossary?
[Note: TOC, Index, and Glossary are linked at bottom of sidebar on many web pages]

4. When you find a problem with using JMRI, how often do you [scale: “every time” to “almost never”]:
 a. Report a problem I find in JMRI to the user group?
 b. Work around a problem rather than reporting it?
[Note: PLEASE report problems - including in documentation - developers really want to find and fix them!]

5. Is there a specific topic (or topics) you would like to see more help on?
[Note: volunteers always accepted for working on these!]

6. Other general comments about help or the web site
[Thanks! We are all part of the user community.]
————-

What do you think?  I know there are two points of view on how to respond to the answers to a survey such as this:  if few answers or little interest in the help/web system, (1) why bother spending time to update it (2) spend a lot more time to make it better (but how?).  Either way, it would be useful to know what people think, and whether they think enough to respond.

Thanks,
Jerry


Bob Jacobsen
 

I think this is a good idea.

We used the Yahoo poll a couple times. It was really clear that the answers had a huge attention bias and a large selection bias, so you couldn’t really count on any numerical percentages, etc. But’s handy to find the full range of responses.

A couple of suggestions on the questions below.

Bob

————
1. When you are working with JMRI, do you generally have access to the internet (on a smartphone, pad, or computer) to get help and other information?
[Note: not necessary, but desirable as access to the user forum requires access to the internet]

2. How often do you [on a scale from "almost every time I use JMRI" to "almost never"]:
a. Use Help from a JMRI window?
b. Access the website JMRI.org to get help?
c. Ask questions in this user forum as my primary source of help?
d. Use other web resources (blogs, YouTube, etc.) for help?
[Note: all these resources are constantly being updated so check back often if you don’t find what you want the first time.]

3. Do you [scale: “every time I use help resources” to “almost never” and “didn’t know this existed”]:
a. Find what I need in the help/web system?
b. Find errors in the help/web information?
c. Use the JMRI Table of Contents to find things?
d. Use the JMRI Index to find things?
h. Use the JMRI Glossary?
I think it would be good to ask about using the search window here. “Use the JMRI search bar or a search engine?"

[Note: TOC, Index, and Glossary are linked at bottom of sidebar on many web pages]

4. When you find a problem with using JMRI, how often do you [scale: “every time” to “almost never”]:
a. Report a problem I find in JMRI to the user group?
b. Work around a problem rather than reporting it?
I would invert the order, so (a) is work around, and then perhaps add “(c) Report a problem I find in the JMRI Issues List”, perhaps linking that.

[Note: PLEASE report problems - including in documentation - developers really want to find and fix them!]
“Please tell us what version of JMRI you’re using” (maybe the choices are “before 4.12”, “4.12”, “4.13.something through 4.16”, 4.17.something, 4.18, 4.19.something, 4.20, "4.21.something”, "a version I build myself”)

5. Is there a specific topic (or topics) you would like to see more help on?
[Note: volunteers always accepted for working on these!]

6. Other general comments about help or the web site
[Thanks! We are all part of the user community.]
————-

What do you think? I know there are two points of view on how to respond to the answers to a survey such as this: if few answers or little interest in the help/web system, (1) why bother spending time to update it (2) spend a lot more time to make it better (but how?). Either way, it would be useful to know what people think, and whether they think enough to respond.

Thanks,
Jerry


Bob Jacobsen
@BobJacobsen


Peter Ulvestad
 

I believe the current high numbers of posts to the users group are due to the number of people staying home. Plus there are a few members who are creating a high number of posts.
Polls for the users group have been deactivated for members to start(I assume the same on this group). I for one am not a fan of polls and normally wont do them. I find in most cases (not saying yours), they are written in a way to solicit the responses the poster wants.

I think 2 things could be done to help.
1) A write-up on how to use the Search function on the JMRI website.
2) A write-up on how the use the help feature in JMRI. (both window and general help).

Both could also be made posts and pinned in the users group as well as being on the website.

--
Peter Ulvestad

JMRI Users Group Moderator - http://www.jmri.org ( http://www.jmri.org )
Tam Valley Group Moderator - https://tamvalleydepot.com/ ( http://tamvalleydepot.com/ )
Sprog-DCC Group Moderator - http://www.sprog-dcc.co.uk/ ( http://www.sprog-dcc.co.uk/ )
Edmonton Model Railroad Association - http://www.emra.club/


whmvd
 

There is also a FAQ page on the JMRI site that screams for content (it has virtually none). Some of the things that come back often (like Dave Heap's responses to connection problems) would be well served with a faq-entry instead, so a simple link in response would give the person with the problem the exact, right response instead of people paraphrasing Dave's (I've been guilty of that). Also: if things change, there would be no outdated information for people to find in the group, just links to current information.

Wouter


On Sat, 4 Jul 2020 at 16:57, Peter Ulvestad <ulvestad@...> wrote:
I believe the current high numbers of posts to the users group are due to the number of people staying home. Plus there are a few members who are creating a high number of posts.
Polls for the users group have been deactivated for members to start(I assume the same on this group). I for one am not a fan of polls and normally wont do them. I find in most cases (not saying yours), they are written in a way to solicit the responses the poster wants.

I think 2 things could be done to help.
1) A write-up on how to use the Search function on the JMRI website.
2) A write-up on how the use the help feature in JMRI. (both window and general help).

Both could also be made posts and pinned in the users group as well as being on the website.

--
Peter Ulvestad

JMRI Users Group Moderator - http://www.jmri.org ( http://www.jmri.org )
Tam Valley Group Moderator - https://tamvalleydepot.com/ ( http://tamvalleydepot.com/ )
Sprog-DCC Group Moderator - http://www.sprog-dcc.co.uk/ ( http://www.sprog-dcc.co.uk/ )
Edmonton Model Railroad Association - http://www.emra.club/




whmvd
 


On Sat, 4 Jul 2020 at 17:02, Wouter van Doorn <vandoornw@...> wrote:
There is also a FAQ page on the JMRI site that screams for content (it has virtually none). Some of the things that come back often (like Dave Heap's responses to connection problems) would be well served with a faq-entry instead, so a simple link in response would give the person with the problem the exact, right response instead of people paraphrasing Dave's (I've been guilty of that). Also: if things change, there would be no outdated information for people to find in the group, just links to current information.

Wouter

On Sat, 4 Jul 2020 at 16:57, Peter Ulvestad <ulvestad@...> wrote:
I believe the current high numbers of posts to the users group are due to the number of people staying home. Plus there are a few members who are creating a high number of posts.
Polls for the users group have been deactivated for members to start(I assume the same on this group). I for one am not a fan of polls and normally wont do them. I find in most cases (not saying yours), they are written in a way to solicit the responses the poster wants.

I think 2 things could be done to help.
1) A write-up on how to use the Search function on the JMRI website.
2) A write-up on how the use the help feature in JMRI. (both window and general help).

Both could also be made posts and pinned in the users group as well as being on the website.

--
Peter Ulvestad

JMRI Users Group Moderator - http://www.jmri.org ( http://www.jmri.org )
Tam Valley Group Moderator - https://tamvalleydepot.com/ ( http://tamvalleydepot.com/ )
Sprog-DCC Group Moderator - http://www.sprog-dcc.co.uk/ ( http://www.sprog-dcc.co.uk/ )
Edmonton Model Railroad Association - http://www.emra.club/




Peter Ulvestad
 

And thanks Jerry for taking on the major task of cleaning up the website.

--
Peter Ulvestad

JMRI Users Group Moderator - http://www.jmri.org ( http://www.jmri.org )
Tam Valley Group Moderator - https://tamvalleydepot.com/ ( http://tamvalleydepot.com/ )
Sprog-DCC Group Moderator - http://www.sprog-dcc.co.uk/ ( http://www.sprog-dcc.co.uk/ )
Edmonton Model Railroad Association - http://www.emra.club/


JerryG
 

Wouter - 

“Screaming for content” may be an understatement.  OTOH, it took me a while to figure out where it was linked so likely very few people ever find it.  I note that there are also FAQs for PanelPro and DecoderPro with some content.  Perhaps we need a JMRI FAQ?  A couple of the first things to add would be Peter’s suggestions for how to use search and help within JMRI.

Peter -

Since I’m not a Java developer (although I’ve had some really excellent ones working for me at various times), adding to the documentation effort is one way I can contribute to this incredible community effort.  Of course, I hope I’m not the only one interested in providing help documentation...it would take a really long time to do an update.  Feel free to suggest pages that need a high priority review, such as the Setup FAQ page.

As to a user survey, I’m still in favor of doing that.  I’ve been in this field long enough to know how scary it sometimes is to learn what the users really think of your work, but then again, where would we be without users?

Is it possible to allow a single person to do a poll (or somehow give temporary privileges to do that)?

And thank YOU for your excellent work (and patience) in managing the user forum (among other things).

Jerry


Peter Ulvestad
 

Jerry,

Groups.io is different from yahoo in that the moderators can't see settings they can't change, but I really doubt that a single person could be set up to do a poll.
Any group changes would have to be done by one of the 5 group owners. Bob J or Matt Harris are probably the ones who would do it. I'm guessing open making polls to everyone then closing it afterwards would be the easiest.

--
Peter Ulvestad

JMRI Users Group Moderator - http://www.jmri.org ( http://www.jmri.org )
Tam Valley Group Moderator - https://tamvalleydepot.com/ ( http://tamvalleydepot.com/ )
Sprog-DCC Group Moderator - http://www.sprog-dcc.co.uk/ ( http://www.sprog-dcc.co.uk/ )
Edmonton Model Railroad Association - http://www.emra.club/


JerryG
 

It's been a while since I used Survey Monkey, but I created a survey that does exactly what I want - and then discovered they want $32 a month to send it!  If I remove all the interesting questions (including those with a comment box), it is free.  Sheesh.

Then I remembered I had an old Qualtrics "free" account....

PLEASE TAKE THIS SURVEY (estimated to take 4 minutes), and give me any suggestions before I put link in the user forum:

https://qtrial2010.qualtrics.com/jfe/form/SV_9FwkN8z8mCUh6lf

Thanks, Jerry


danielb987
 

I took the survey and it seems good.

Daniel

2020-07-06 04:34 skrev JerryG via groups.io:

It's been a while since I used Survey Monkey, but I created a survey
that does exactly what I want - and then discovered they want $32 a
month to send it! If I remove all the interesting questions
(including those with a comment box), it is free. Sheesh.
Then I remembered I had an old Qualtrics "free" account....
PLEASE TAKE THIS SURVEY (estimated to take 4 minutes), and give me any
suggestions before I put link in the user forum:
https://qtrial2010.qualtrics.com/jfe/form/SV_9FwkN8z8mCUh6lf
Thanks, Jerry
Links:
------
[1] https://jmri-developers.groups.io/g/jmri/message/3853
[2] https://groups.io/mt/75257846/1303822
[3] https://jmri-developers.groups.io/g/jmri/post
[4] https://groups.io/g/jmri-developers/mutehashtag/webhelpupdate
[5] https://jmri-developers.groups.io/g/jmri/editsub/1303822
[6] https://jmri-developers.groups.io/g/jmri/leave/defanged


Bill Chown
 

Took the survey, liked it.
Small typo: JMR for JMRI - "Do you use other forums where JMR is discussed?"

Bill



On Sunday, July 5, 2020, 07:34:51 PM PDT, JerryG via groups.io <jerryg2003@...> wrote:


It's been a while since I used Survey Monkey, but I created a survey that does exactly what I want - and then discovered they want $32 a month to send it!  If I remove all the interesting questions (including those with a comment box), it is free.  Sheesh.

Then I remembered I had an old Qualtrics "free" account....

PLEASE TAKE THIS SURVEY (estimated to take 4 minutes), and give me any suggestions before I put link in the user forum:

https://qtrial2010.qualtrics.com/jfe/form/SV_9FwkN8z8mCUh6lf

Thanks, Jerry


JerryG
 

Thanks for "testing" and comments.  Results from 9 of you posted to "files" under JerryG directory.  Still time to try it out.  I'll plan on posting to user group at beginning of next week.

Jerry


Andrew Crosland
 

I have just done it. I'm more of a developer than a user so the questions didn't always map to my usage/expectations.

--
Andrew Crosland


Peter Ulvestad
 

The results show that this isn't really the target group. I guess it will be interesting to see what the users group responses are.

--
Peter Ulvestad

JMRI Users Group Moderator - http://www.jmri.org ( http://www.jmri.org )
Tam Valley Group Moderator - https://tamvalleydepot.com/ ( http://tamvalleydepot.com/ )
Sprog-DCC Group Moderator - http://www.sprog-dcc.co.uk/ ( http://www.sprog-dcc.co.uk/ )
Edmonton Model Railroad Association - http://www.emra.club/


Pete Cressman
 

Very good survey. 
Possible addition:
"Is there topic (or topics) that has inaccurate information or none at all?


On Sunday, July 5, 2020, 07:36:32 PM PDT, JerryG via groups.io <jerryg2003@...> wrote:


It's been a while since I used Survey Monkey, but I created a survey that does exactly what I want - and then discovered they want $32 a month to send it!  If I remove all the interesting questions (including those with a comment box), it is free.  Sheesh.

Then I remembered I had an old Qualtrics "free" account....

PLEASE TAKE THIS SURVEY (estimated to take 4 minutes), and give me any suggestions before I put link in the user forum:

https://qtrial2010.qualtrics.com/jfe/form/SV_9FwkN8z8mCUh6lf

Thanks, Jerry


JerryG
 

72 full responses (with several dozen comments) in first 24 hours that survey has been available on user forum.  Also got 18 responses from this list....


JerryG
 

Over 180 responses to date.


JerryG
 

The results are in.  Almost 160 usable surveys (not counting developers who did the pre-test) were filed over a two week period ending 7/23 from over a dozen countries, (60%+ US, from 34 states).  I put my results document into file folder JerryG in this forum, and will distribute a similar version (without my intermediate work sheets) to the Users Group.  Please scan the almost 100 comments that people wrote in (several key ones summarized below).  Post to thread with your comments.

 

Here are a few highlights that I picked out:

90% of respondents have access to internet when working with JMRI.

 

76% use Help from a JMRI window at least some of the time.

81% access the web site for help at least some of the time.

62% ask questions in the user forum (which is interesting since you would likely have to be a user of the forum to even know about the survey - lots of lurkers).

Only 30% ask a friend for help!

 

90% find the help they need via Help/web at least some of the time but only 37% say “most of the time.”

33% find errors in help/web some of the time.

62% use the Table of Contents.

63% use the Index.

Only 34% use the Glossary.

 

29% are on 4.19x.

32% are on 4.20 (note that this was only 1-2 weeks after 4.20 came out).

Only 30% report updating to production release soon after it comes out.

37% report usually installing test releases. 

 

56% will report a problem to the user group at least some of the time

18% report a problem to the Issues List in Github

 

Collectively, they reported using over 20 other forums.

 

Some key write-in responses (edited, grouped, summarized):

Great software, appreciate all your work, "best community maintained s/w I have seen"  (more than a dozen kudos).

Have difficulty finding content, needs more cross references, etc.

Explanations not sufficiently geared to beginners (should be “like we were 5th graders”).

Need “getting started,” step-by-step, how the pieces fit together

Indicate what version the screen shots are from and that the text refers to (much is out of date or inconsistent).

Nice to have many ways to do essentially the same thing - but need comparison of pros/cons of each (e.g Logix vs scripts vs Routes, LRoutes vs Routes).

 

For the user group: Those who know nothing about a problem should refrain from saying or requesting info as if they do.

 

So I would say this was affirming of the value of help info, although pointing out the interest in simpler explanations and more “integrating” step-by-step pages (like the one Dave Sands just did for PanelPro).  The idea of labeling each page or picture with what release it refers to might help to avoid confusion as pages get out of date (while not exactly a solution to this issue,  we have just added a line to the standard Footer that provides a pointer to the page history).  So definitely some things to think about as we create/update help info.

 

I’ll post a version of this to the Users’ Forum in a couple of days.

 

Jerry

 

P.S.  Is there anything useful we can do with the names/emails of the people who provided them?  About 60.