Creating Moodle Quizzes: Good Questions First!

Listen with webreader

The most difficult part of creating a Moodle quiz isn’t using Moodle.  It is writing the questions.  First, write the good questions. Then create the Moodle quiz.*

Test questions – regardless of delivery method – must meet two important criteria:

  • They must have validity. This means they test what you say they’re testing.  If you’re qualifying someone to do CPR, you must provide valid testing that truly measures a person’s ability to perform CPR in an emergency.  You don’t want to be measuring a person’s ability to memorize an acronym or to perform a tracheotomy.
  • They must be reliable. This means that if someone takes theA thermometer with both F and C scales.  To be valid in each, the scales must be correctly spaced and aligned with each other, as well as the position of the mercury tube. test today, he’ll answer the same as he did yesterday – or will tomorrow.  It also means that the person, his background, the time of day, the method of delivery, etc., will not affect the answers.  The only thing that should alter the answers is the actual understanding of the subject.

Think of each of your questions as a thermometer.  To be valid, it must measure temperature, not weight or decibels.  To be reliable, it must read 32°C when it is in fact, 32°C, regardless of the relative humidity or other ambient conditions.

How do you create test questions that are both valid and reliable?

  • Test the Topic
  • Keep a Poker Face
  • Mix It Up

Test the Topic, Not the Ability to Do Word Puzzles

Bad Question #1: Don’t confound things by asking two questions in one. Don’t get tricky. Don’t ask irrelevant questions.  Don’t write questions like this one:

Which of the following statements is not true?

A. The grass is green and the sky is blue
B. Neither grass nor the sky is blue
C. The grass is green or the sky is blue, but not both
D. All of the above
E. None of the above

Seriously? I don’t even know the right answer to this question.  I’m not even sure there is a right answer!

Better Question #1. If you’re teaching the science of color, you want to ask questions about why grass appears green or the sky appears blue. You don’t want to test the ability to reason through a maze of “not”, “or”, “and”, “all”, “none”…

What explains the appearance of a blue sky of varying shades during the day?

A. Scattering of sunlight by air molecules
B. Oxygen molecules are blue
C. The sun lightens the darkness of space, making it appear blue
D. The blue color is not real; it is an optical illusion

Better Question #2. If, on the other hand, you’re teaching logic, you’ll want to make each choice require the student to reason through it.

Which of the following statements is true? Check all that apply.

A. Grass is green and the sky is blue.
B. Grass is green or the sky is blue.
C. Grass is green and/or the sky is blue.
D. Neither grass nor the sky is blue.
E. Neither grass nor the sky is green.

Keep a Poker Face.  Don’t Give Away the Right Answers.

Novice test writers often give away the correct answer by putting too much into it.  It’s easy to get excited about a concept or just want to really get the point across.  It’s hard to put that much effort into the wrong answers, so they are often silly afterthoughts. This reduces the validity of the test by making it obvious what the right answer is, even to someone who knows nothing about the subject.

Bad Question #2: Too long to be wrong…(and confounding by including so many terms in one choice)

Which one of the following is true?

A. Good leadership is a matter of opinion.
B. Poor leadership (no demonstration of values, inconsistent, poor decision making, unavailable or unapproachable) in the workplace can result in poor performance for the entire organization, at all levels.
C. It is difficult to define leadership.
D. Leadership is not necessary if the team is solid.

Bad Question #3: Silly wrong answers…

Choose three of the five core emotions:

A. Joy
B. Beer
C. Anger
D. Love
E. Ice Cream
F. Sadness
G. Football
H. Fear
I. Baseball
J. Shame

Better Question. The wrong answers should not only be plausible, but they should represent some of the most common misunderstandings surrounding the topic. This will provide you with true information about how well the students understood your lesson and gives you an opportunity to reinforce the concept with your feedback.

From the following list, select the five core emotions:

A. Anger
B. Shame
C. Fear
D. Love
E. Hate
F. Joy
G. Depression
H. Sadness
I. Laziness
J. Pride

The letter “B” might be your favorite choice, but make sure it is not always the correct choice.  Vary the position of the correct answers in your questions.  Also, be grammatically consistent; don’t give the correct answer away by making it the only one that flows as a sentence with the question.

For more examples of how to improve the wording of a question, check out the Guide to Objective Tests at the Computer Aided Assessment Centre.

Mix It Up to Keep Them Thinking

Limit True/False questions. They can become hypnotic and don’t afford you with the ability to learn specifics of what the student doesn’t understand.

Open ended questions (fill in the blank, short answer, and essay) can be hard to grade. They should never be used unless just one person grades all submissions and that person is an expert in the field. The last thing you want is a non-expert searching for “key words” the way a computer would. That would be testing the student’s ability to win Buzz Word Bingo, not whether there was an understanding of the topic.  After all, it takes true understanding to paraphrase.

Use multiple choice questions with varying formats. Use “check all that apply”, “check the one that does not apply”, “choose only one”, and even an introductory scenario with a few follow-on multiple choices. Use common misconceptions as wrong answers.

Be creative (not tricky!) with matching questions. Have more possible answers than questions (uneven left and right columns) so it doesn’t become a process of elimination.  Include common misconceptions as possible matches.

Test Designer has a great reference page for what type of question to use, depending upon your intent.

Stay tuned for an upcoming post on the merits and pitfalls of shuffling questions and choices.

*For more on using Moodle quizzes and testing in a business environment:

Posted in Good Business, More Good Stuff - Moodle Tips, Computer Tricks, etc. | Tagged , , , | Leave a comment

You’ll Save More Than Trees, Part II: Everyday Practices

Listen with webreader

Like fashion of the 1970s, there are some late 20th century trends that should be apologized for and forgotten. We are all better off moving on from them.

Or maybe we should be moving backward.  My grandmothers grew vegetables in their gardens and took cloth bags to the market.  They wrapped gifts in plain brown paper or nothing at all.  Sounds a lot like today… They didn’t recycle – let alone throw things away – they reused everything.

Not that long ago, offices (and even police stations) ran smoothly on file cabinets filled with index cards. A “bank book” was the size of a passport and covered years of transactions. Compare that to a bank statement today where 29 days worth of transactions requires 5 pages, printed front and back.  When information was stored on index cards and written out by hand, people were more selective about what information they maintained.

Be green and put money in the bank at the same time.

Paper wasn’t the only thing they saved.  They saved time by not having to search through so much stuff. And as the adage goes “time is money”.

Today, it is so easy to print, fax, copy, and save that we are buried in piles of nothingness.  Here are some ideas on how we can move forward by taking a lesson from the past.

  1. Keep your policies and procedures online. You should do this anyway, for document control purposes.  To help people follow them, create job aids – posters, quick reference guides, badges, mugs, you name it – with important reminders. At you can create your own or select from their sizeable collection.  Titles might include:
    • Leaving For the Day? Turn it off.  Lock it up.
    • Protecting Company Assets
    • Protecting Intellectual Property
    • Identifying Stroke in a Co-worker
    • Before You Call IT, Did You…(a troubleshooting guide)
    • Avoiding Colds and Flu This Season
  2. Encourage employees to find information online instead of photocopying everything.  You’d be surprised how many owners’ manuals, users’ guides, and even recipes are online!
    • Work with your IT department to “allow” these websites (like and tell employees to share their recipes and photos electronically.
    • Many larger companies are setting up internal “social networking” sites for employees to do just that. It is possible for a small business to as well, using Facebook, for instance.
    • Not only will you save paper, you’ll also save ink, money on parts and maintenance, time wasted while the machine is being serviced, and extend the life of your copiers/printers.
    • You will reduce landfill needs, disposal costs of hazardous materials (all electronics, including batteries, need to be disposed of carefully), and all of the costs associated with the production and shipping of a replacement machine.
  3. Encourage your employees to save, not print, emails.
  4. Store your accounting and tax documents electronically. You’ll reduce the time you and your bookkeeper spend transcribing data (it’s already online) and you’ll have a reliable “paper trail“.  The US Government began requiring electronic payments to all contractors more than a decade ago.  If they can do it, so you can you!
  5. Stop using fax machines and accept electronic signatures.
    • If you need receipts from your employees, have them scanned.  Scanned documents have a longer life than those horrible little shiny paper things.
    • Adobe offers a digital signature on PDF forms.
    • Have a lawyer draw up a paragraph that essentially says “by typing your name here you are placing your signature on this document”.
    • These documents can be stored easily and they don’t require faxing or mailing.
  6. Provide filtered water at the office. Water is healthy and it’s good to make it accessible so everyone will drink enough of it.
    • Offering bottled water for sale or for free is nice, but not enough.  Americans use more plastic and throw away more bottles than the rest of the world combined.  It’s shameful.
    • Install inline filters (under $100) and keep a supply of new filters on hand.
    • Order up some stainless or BPA-free Nalgene bottles or porcelain mugs (which work for coffee, too) with your logo or company mission on them. Give them out to employees or sell them, with proceeds going to a local charity.
  7. Use wireless components only when necessary. If a DSL cable or USB mouse is feasible, use it.  Wireless components need batteries; batteries create hazardous waste, cost money to buy, and use resources to produce.  If you must use batteries, use rechargeable ones.
  8. If your employees wear uniforms, consider switching to washable fabrics. Dry cleaning fluid is a dreadful thing and it takes time to drop off and pick up clothes.  If you must use dry cleaning, choose one such as Green Earth Cleaning.
  9. For those times you really can’t do without paper materials (such as advertising and invitations), make sure your printer uses responsible papers and inks.
    • The Greg Barber Company is an eco-friendly printer.  Check out the seed paper invitations they did for the NFL.  Very cool idea!  They also have biodegradable banners, for your next company picnic or trade show.
    • Another alternative is elephant dung paper.  A search of that phrase shows that you can buy it just about any where!
  10. Travel with the environment in mind. If your business is in the New York City metropolitan area or you travel there for any reason, check out Firefly Concierge.  This is a service dedicated to sustainable living, working, and visiting in The Big Apple. If you’re traveling to other locations, search on “green travel” for similar services.  What happens in Vegas not only stays there, it also impacts the environment there.

Additional Reading

For more ideas, more posts:

To learn more about ways to reduce paper usage and to read some of the stunning statistics on waste, check out these sites:

Sites to help in your sustainability efforts:

Posted in Good Business, Lean & Green | Tagged , , , , , , , , , , , | Leave a comment