Blackboard Satisfaction Survey Results – Dislikes

May 13, 2011

Taking a look at the second of the three open-ended questions posed in our Blackboard Faculty Satisfaction Survey, I expected some grim reading and certainly got it.  The question: “What do you like least about Blackboard?”

246 faculty responded to this question (out of 301 total survey respondents), although a half-dozen responses were some variant on “N/A”.  In an attempt to process what themes might be present in a mass of text, I like to create a Wordle and view the word frequency use graphically.  (To see any of the smaller text entries, you will want to click the thumbnail image below and see the larger version.)

What do you like least about Blackboard? wordle

Although the focus on students is still obvious, it’s clear even from the Wordle that some other recurring themes arise.  Since we made the (very visually and functionally significant) upgrade from Blackboard version 8 to 9.1 just prior to this term, I’m not surprised to see that “version” cropped up a great deal.  I’m expecting that major change is also responsible for the importance of “new” and “use”, but only the details of the comments themselves will tell for sure.

There were 34 comments (almost 14%) saying that faculty disliked the grading functions.  Given that grade functionality was also among the most liked functions, I’m going to draw this conclusion: Faculty like being able to distribute grades via Blackboard, while disliking the actual procedures to do so.  Not exactly an earth-shaking conclusion, but it is nice to base that on actual data rather than pulling an Obi-Wan and just trusting my feelings.  (And, should any readers NOT get the whole Star Wars “trust your feelings, Luke” reference, perhaps they need to stop reading my blog posts.  Alternately, go see some movies.)

Survey results also yielded 13 comments to the effect that our Blackboard system is slow.  Now some of those are in reference to specific subsystems (such as “entering grades”), and others put qualifiers (such as “when using my dial-up connection” or “but that’s through my satellite Internet”), but that’s quite at odds with my own use experiences.  Of course, I personally use the system only from on-campus…  My take-away:  Start doing some off-campus system performance testing.  Very significant changes were made to the platform our Blackboard system is running on (such as using virtual front-end servers, but a whole host of other changes as well) when we made the switch to Bb9.1 in January, so perhaps there are some technical speed issues that I am simply unaware of that is impacting faculty.

I have to send out kudos to one of our support techs, because in these comments of all places a faculty member felt that Chris Norcross has given him exemplary support.  Good job, Chris!  Actually, in the “what faculty like most” comments there was also a suggestion that our support techs be cloned so that we can have a 24/7 supply of them available.  As soon as the clone pods become available, Chris gets to go first.

A good many comments involved frustration at how Blackboard is not fully functional with some versions of some browsers.  I really feel for the faculty on this one; in today’s software environment there are changes being made to web browser software on a very frequent basis, so it is nearly impossible to not end up upgrading away from a browser version that works… should you be so fortunate as to find one that works fully in the first place.  Not to make this out to be more than it really is, because most functions of Blackboard work well in most versions of most browsers, but some of the subsystems that faculty have to regularly make use of are more complex than what the average Blackboard user (read: student) has to do.  Test construction, the Grade Center, all the controls for adding content, this stuff is complex and puts a large burden on the browser program to process it properly; the one lesson I’ve learned over the years supporting online work is that standards aren’t, and nothing works consistently across all browsers.

All that being said, this makes it all the more important that we keep updating the version of Blackboard used at Palomar.  Several comments complained that Internet Explorer 9 or Firefox 4 were not officially supported browsers, but the way browsers get added to the support matrix is through testing on new versions.  So, faculty, there’s the dilemma: do we try to lag behind to stay with known to work versions (nothings perfect, but the sometimes the known bugs are easier to live with than at other times), or do we endeavor to keep as updated as possible to try to have maximum compatability?  One thing we in Academic Technology have tried to do is never do a non-critical upgrade during a semester, so even during the Spring 2011 term there would have been no way to get versions of Blackboard that would have supported browsers released over the Spring.  I’m not sure what the correct answer to that question is, but hopefully our “upgrade between terms” model will strike the best happy medium possible.  Suggestions on this topic are certainly welcome, of course.

There are a few comments along the lines of “I don’t like that Blackboard cannot do ____.”  In many cases, it actually can, so I’ll be gathering up those comments (and I expect to get more such from the final open-ended question on “what would you change about Blackboard”) and respond to them in a future post.  I remember speaking with a Microsoft employee after a presentation at the Office 2007 launch event I attended, where he described some experiences he’d had in surveying users.  The most memorable story (for me, at least) was his discussion with a panel of Math professors who were bitterly complaining that there was no function in Word for putting in equations.  He admitted that he did finally tell them about how the equation editor had been a component of Word for years; to him it was an obvious function, but whole swathes of users were unaware of it.  Apparently something of the same nature is going on with Blackboard users here.

That wraps up my current thoughts on what faculty “like least about Blackboard” at the moment.  As I mentioned above, I’ll be revisiting some of these issues once I’ve had a chance to process the final open-ended question on what faculty would change.


Blackboard Satisfaction Survey Results – Likes

May 12, 2011

Taking a look at the first of the three open-ended questions posed in our Blackboard Faculty Satisfaction Survey hasn’t been a huge surprise, which is a good thing.  (I’d hate to think my feel for faculty opinions is too far off of reality.)  The question: “What do you like best about Blackboard?”

259 faculty responded to this question (out of 301 total survey respondents), which makes for quite a lot of commentary reading.  In an attempt to process what themes might be present in a mass of text, I like to create a Wordle and view the word frequency use graphically.  (To see any of the smaller text entries, you will want to click the thumbnail image below and see the larger version.)

Wordle of what faculty like about Blackboard

It was gratifying to see what the largest word (and therefore the presumed primary focus of the comments) worked out to be.

Looking over the actual comments, I noticed a high frequency of comments (nearly 10%, 24 to be specific) mentioning the “ease of use” of Blackboard.  I’m not too surprised to see quite a few comments liking “grades” or other grade-related functions such as “Gradebook”.  Keep in mind that this is what faculty like about Blackboard, not necessarily that it’s what is easy to do in Blackboard.

Not all the comments are sunny and bright, of course.  One professor writes “Not sure there is anything that stands out that can’t be done elsewhere better.”  This is a valid point; the best analogy I’ve ever heard used to  describe Blackboard came from a support tech at some past conference: “Blackboard is Tupperware.”  All it is ever going to do is hold the material faculty put into it, and by trying to support all the different things that faculty want at the same time it can be difficult to do any one specific thing the best.  Certainly there are better discussion board systems, better content management systems, etc.  I’m not sure that there are systems that do significantly better that do as much (in potential, at least) as Blackboard while allowing institutional SIS integration.  The recurring theme I hear when faculty come back to using Blackboard is that although it doesn’t do any one thing the best, it does bundle tools together in a way that is “good enough.”

One particularly glowing comment mentions that the professor “Couldn’t live without it.”  I’ll admit to amazement, when our system is down for upgrades or maintenance, at how many faculty seem to want to interact with our Blackboard system even in what is typically thought of as “down time” between semesters.  I’ve had a professor want to know if I really must have the system offline on New Year’s Eve… sheesh, go party and I’ll try to have the system back online before morning!

One aspect that comes through looking at the comments directly that isn’t apparent from the Wordle is how often faculty referred to the discussion board.  (Seems folks don’t know how to refer to it: DB, disc. brd, forums, there were too many ways to describe that function to become prominent in the image.)  We generally get poor attendance at training workshops covering the discussion board though, so perhaps faculty feel like they truly have a good grasp on how to use that tool; apparently they like the function well enough.

That pretty much wraps up my thoughts on what faculty “like best about Blackboard” at the moment.  I’m sure I’ll be revisiting some of these issues once I’ve had a chance to ponder the other two open-ended questions on what faculty like least and what they would change.


Blackboard Satisfaction Survey Results – First Look

May 11, 2011

As part of the Palomar College Academic Technology department’s efforts to measure success (and thus meet some reporting requirements), we invited the Palomar Faculty who were assigned to available Blackboard courses this term to take a satisfaction survey.

We (like so many others) used SurveyMonkey to conduct the survey, which worked very well.  The invitation went out to 601 faculty members, of which 301 actually responded; I cannot say how happy I am with that sort of survey return rate!  The survey period ended last Friday afternoon, so I’m finally able to sit down and wrestle with the results.  For a first look at those results, I’m going to post publicly the more “aggregate” data; later posts will deal with the results of the (more important, to my mind) open ended questions asked at the end of the survey.

The image describes the detailed results, but the gist of the first question set tells me that although 79.4% of faculty surveyed are satisfied with Blackboard overall, far less are satisfied with the ease of use, with 17.3% being unsatisfied on that issue.

Survey Question 1 data results

Again the details are in the image, but next up was an opinion-style set asking about ease of student login, course material setup, and test setup.  Unsurprisingly, only 30.5% of faculty agreed that tests are easy to set up.  (Frankly I’m amazed that many responded such. I don’t consider the process to be easy, and I teach others how to do it!)  84% of faculty agreed that student login was easy… I’ve no idea how we could make it easier than the current “same as you used to enroll in classes” though.

Survey detailed data results

Finishing out the agree/disagree questions, 68.9% of faculty agreed that Blackboard has improved their communication with students, while 66.2% agreed that Blackboard created new opportunities to teach and learn.  I am thrilled to death by these results.

Detailed survey results

I found the responses given to “how many years have you been using Blackboard at Palomar College” to be interesting, because according to our results we’ve had roughly the same number of faculty start using it across the last five years.  Also we have faculty claiming to have been using Blackboard here at Palomar for longer than Palomar has used Blackboard… but that’s likely just a matter of remembering years wrong.

Detailed survey results

We finished up the objective questions by asking which Blackboard features faculty use.  My take-away from this is that either I have no idea how faculty actually use their Blackboard courses, or that faculty don’t know the proper names of the tools and components in their course sites.  (I’m hoping it’s the latter, because I just cannot believe that only 40.3% of faculty use the Item.)

Detailed survey data

The survey finished with three open questions:

  1. What do you like best about Blackboard?
  2. What do you like least about Blackboard?
  3. If you could, what would you change about Blackboard?

I’ll delve into some of the information from these responses in other posts, but to my mind that’s where the real meat of the useful information will be.

So, looking back on all this data, if you have any thoughts you’d like to share, please notice that prominent box at the bottom of the page to leave comments.


Don’t Shoot the Messenger

May 6, 2011

We’ve just had occasion to change the way part of Palomar’s Blackboard Learn system functions, so I wanted to explain why the system is behaving the way it now is.

If you’ve posted an Announcement in the new Blackboard system, you’ve likely seen that check box for “Override User Notification Settings” where previous versions had a “Send Email” box.  (I don’t propose to go into the reasons for that change; if you’re curious then feel free to get in touch with me via email.)  If you check that box, when you post your announcement in the course a notice gets sent to all your students telling them that there is an announcement in your course they should look at.  If you do not check that box, students would not receive such a notice.

Woah, “would not”? Is that what changed?

Yes.  The new behavior in our Blackboard system is for students to receive a nightly notice (called a Digest Notification, sent around 11 PM) about ALL announcements made in their courses.

Functionally what this means is if you post five announcements during the day, but only one of them you check the “Override User Notification Settings” box, your students would receive two notices.  One would be about just the announcement where you checked the box which would go out immediately, the other would be about the rest sent near the end of the day.

I could go into a long-winded explanation about exactly why we had to change the system behavior, (short answer: to work around a bug) but I’ll spare everyone that confusion.  Bottom line, more notices are now going out to students about announcements than ever before.