Fast and Efficient Static Randomisation via Sagemath, Generators, and Decorators

Björn Rüffer, The University of Newcastle, Australia

E-Assessment in Mathematical Sciences (EAMS) 2020, Newcastle University, UK

25 June 2020

Features of this presentation

  • use SageMath (and jupyter notebooks)
  • generate static output
  • be agnostic towards output formats and question types
  • typeset and preview formulas with $\LaTeX$

Outline

  • Three examples
  • One tutorial about workflow ingredients

Example: Print output

Example: BlackBoard Multiple Choice

Example: Plain text for a vision impaired student

The tutorial

Give a man a fish and you feed him for a day. Teach him how to fish and you feed him for his life time.

In this very spirit, I will not share code, but show you how you can make your own.

Here are the ingredients:

  1. Lists and generator expressions
  2. LaTeX in previews and outputs
  3. Generators
  4. Decorators

Lists and generator expressions

Lists

In [1]:
a = (1,2,3,4,5,6,7,8,9,10) # a tuple
b = [1,2,3] # a list

c = ['Text or numbers, or bools. Any type, really.', 42, True, a, b]

d = [a,b,c] # nested list

display(d) # display produces nicer output in notebooks than print
[(1, 2, 3, 4, 5, 6, 7, 8, 9, 10),
 [1, 2, 3],
 ['Text or numbers, or bools. Any type, really.',
  42,
  True,
  (1, 2, 3, 4, 5, 6, 7, 8, 9, 10),
  [1, 2, 3]]]
In [2]:
e = {1,2,3,3,3,3,3} # a set
display(e)
{1, 2, 3}

Generator expressions

In [3]:
# remember a = (1,2,3,4,5,6,7,8,9,10)
[x^2 for x in a]
Out[3]:
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
In [4]:
question_fragments = (
    ('4+2', '6'),
    ('2+5', '7'),
    ('8-3', '5'),
    ('8-5', '3'),    
)
In [5]:
[f'What is {question}?' for question, answer in question_fragments]
Out[5]:
['What is 4+2?', 'What is 2+5?', 'What is 8-3?', 'What is 8-5?']
In [6]:
answers = {answer for question, answer in question_fragments} # extract answers

def mca(answers, answer):
    'format multiple choice answers as text, highlight the correct answer'
    answers = list(set(answers)) # copy with unique elements
    shuffle(answers) # this is in-place
    answers = answers[:4] # the the first four
    answers += ['None of the others']
    def correct(a):
        if a == answer:
            return '*'
        else:
            return ' '
    return "\n".join(f'  {correct(a)} {chr(65+i)}) {a}' for i,a in enumerate(answers))
print(mca(answers,'7'))
    A) 5
    B) 3
  * C) 7
    D) 6
    E) None of the others
In [7]:
question_db = [f'''What is {question}?\n{mca(answers,answer)}''' for question, answer in question_fragments]
question_db
Out[7]:
['What is 4+2?\n    A) 7\n    B) 5\n    C) 3\n  * D) 6\n    E) None of the others',
 'What is 2+5?\n  * A) 7\n    B) 3\n    C) 6\n    D) 5\n    E) None of the others',
 'What is 8-3?\n  * A) 5\n    B) 3\n    C) 7\n    D) 6\n    E) None of the others',
 'What is 8-5?\n    A) 6\n    B) 7\n    C) 5\n  * D) 3\n    E) None of the others']
In [8]:
[q.split('\n') for q in question_db]
Out[8]:
[['What is 4+2?',
  '    A) 7',
  '    B) 5',
  '    C) 3',
  '  * D) 6',
  '    E) None of the others'],
 ['What is 2+5?',
  '  * A) 7',
  '    B) 3',
  '    C) 6',
  '    D) 5',
  '    E) None of the others'],
 ['What is 8-3?',
  '  * A) 5',
  '    B) 3',
  '    C) 7',
  '    D) 6',
  '    E) None of the others'],
 ['What is 8-5?',
  '    A) 6',
  '    B) 7',
  '    C) 5',
  '  * D) 3',
  '    E) None of the others']]
In [9]:
import pandas as pd
df = pd.DataFrame([q.split('\n') for q in question_db])
df
Out[9]:
0 1 2 3 4 5
0 What is 4+2? A) 7 B) 5 C) 3 * D) 6 E) None of the others
1 What is 2+5? * A) 7 B) 3 C) 6 D) 5 E) None of the others
2 What is 8-3? * A) 5 B) 3 C) 7 D) 6 E) None of the others
3 What is 8-5? A) 6 B) 7 C) 5 * D) 3 E) None of the others
In [10]:
df.to_csv('my_question_database.tsv', sep='\t') # TAB separated output file

! cat my_question_database.tsv # look at the contents of the output file
	0	1	2	3	4	5
0	What is 4+2?	    A) 7	    B) 5	    C) 3	  * D) 6	    E) None of the others
1	What is 2+5?	  * A) 7	    B) 3	    C) 6	    D) 5	    E) None of the others
2	What is 8-3?	  * A) 5	    B) 3	    C) 7	    D) 6	    E) None of the others
3	What is 8-5?	    A) 6	    B) 7	    C) 5	  * D) 3	    E) None of the others

Of course SageMath can do more exciting things like the following:

In [11]:
x = var('x')
p(x) = sum(randint(-10,10)*x^k for k in range(10))
p
Out[11]:
x |--> 8*x^9 + 10*x^8 + 3*x^7 - 10*x^6 - 3*x^5 + 8*x^4 - 7*x^3 + 9*x^2 + 9*x - 5
In [12]:
show(p)
In [13]:
show(diff(p,x))

Handling LaTeX

SageMath will typeset formulas with show(<sage expression>). But sometimes we want to typese more complicated $\LaTeX$ constructs for previewing purposes.

In [14]:
class MATHJAX():
    'take a HTML formated string with embedded latex and typeset it'
    def __init__(self,s):
        self.s = s
    def _repr_html_(self):
        'this is used by display() / show() in your jupyter notebook'
        return '''
            <div>
            <script type="text/javascript" async="" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-MML-AM_CHTML"></script>
            </div>        
            ''' +  self.s
    def __str__(self):
        'this is used by print() and str()'
        return self._repr_html_()
In [15]:
# example
display(MATHJAX(r'Hello, <i>world</i>. \[ \int_a^b f(x) dx. \]'))
Hello, world. \[ \int_a^b f(x) dx. \]
In [16]:
print(MATHJAX(r'Hello, <i>world</i>. \[ \int_a^b f(x) dx. \]'))
            <div>
            <script type="text/javascript" async="" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-MML-AM_CHTML"></script>
            </div>        
            Hello, <i>world</i>. \[ \int_a^b f(x) dx. \]

We can also access the $\LaTeX$ representation of a SageMath expression as a string via latex(<sage expression>).

In [17]:
latex(p(x)) # remember out polynomial?
Out[17]:
8 \, x^{9} + 10 \, x^{8} + 3 \, x^{7} - 10 \, x^{6} - 3 \, x^{5} + 8 \, x^{4} - 7 \, x^{3} + 9 \, x^{2} + 9 \, x - 5

Generators

remember generator expressions?

[x^2 for x in a]

[f'''What is {question}?\n{mca(answers,answer)}''' for question, answer in question_fragments]

[q.split('\n') for q in question_db]

Here's another one

[1/(k+1) for k in range(10)]
# [1, 1/2, 1/3, 1/4, 1/5, 1/6, 1/7, 1/8, 1/9, 1/10]

A question generator

In [18]:
def quiz_1_elementary_products():
    for _ in range(10):
        a, b = randint(1,9), randint(2,10)
        yield rf'What is ${a}\cdot {b}$?', f'{a*b}', f'{a+b}', f'{a-b}', f'{a/b}'
In [19]:
quiz_1_elementary_products()
Out[19]:
<generator object quiz_1_elementary_products at 0x25d20aad0>
In [20]:
question_1_bank = [q for q in quiz_1_elementary_products()]
question_1_bank
Out[20]:
[('What is $4\\cdot 10$?', '40', '14', '-6', '0.4'),
 ('What is $3\\cdot 7$?', '21', '10', '-4', '0.42857142857142855'),
 ('What is $2\\cdot 5$?', '10', '7', '-3', '0.4'),
 ('What is $2\\cdot 4$?', '8', '6', '-2', '0.5'),
 ('What is $6\\cdot 4$?', '24', '10', '2', '1.5'),
 ('What is $9\\cdot 10$?', '90', '19', '-1', '0.9'),
 ('What is $4\\cdot 6$?', '24', '10', '-2', '0.6666666666666666'),
 ('What is $2\\cdot 10$?', '20', '12', '-8', '0.2'),
 ('What is $1\\cdot 7$?', '7', '8', '-6', '0.14285714285714285'),
 ('What is $5\\cdot 3$?', '15', '8', '2', '1.6666666666666667')]

We need a formatter for such lists of questions and four answers.

In [21]:
formatted_questions = [f'{q}\n\n{mca(answers,answers[0])}' 
                            for q, *answers in question_1_bank]
formatted_questions
Out[21]:
['What is $4\\cdot 10$?\n\n  * A) 40\n    B) 0.4\n    C) 14\n    D) -6\n    E) None of the others',
 'What is $3\\cdot 7$?\n\n    A) 0.42857142857142855\n    B) -4\n  * C) 21\n    D) 10\n    E) None of the others',
 'What is $2\\cdot 5$?\n\n    A) 7\n    B) -3\n  * C) 10\n    D) 0.4\n    E) None of the others',
 'What is $2\\cdot 4$?\n\n    A) 0.5\n  * B) 8\n    C) -2\n    D) 6\n    E) None of the others',
 'What is $6\\cdot 4$?\n\n    A) 10\n    B) 1.5\n  * C) 24\n    D) 2\n    E) None of the others',
 'What is $9\\cdot 10$?\n\n    A) 0.9\n  * B) 90\n    C) 19\n    D) -1\n    E) None of the others',
 'What is $4\\cdot 6$?\n\n  * A) 24\n    B) -2\n    C) 10\n    D) 0.6666666666666666\n    E) None of the others',
 'What is $2\\cdot 10$?\n\n    A) -8\n    B) 12\n  * C) 20\n    D) 0.2\n    E) None of the others',
 'What is $1\\cdot 7$?\n\n    A) -6\n    B) 0.14285714285714285\n    C) 8\n  * D) 7\n    E) None of the others',
 'What is $5\\cdot 3$?\n\n  * A) 15\n    B) 8\n    C) 2\n    D) 1.6666666666666667\n    E) None of the others']
In [22]:
for fq in formatted_questions[:2]:
    print(fq,end='\n\n\n')
What is $4\cdot 10$?

  * A) 40
    B) 0.4
    C) 14
    D) -6
    E) None of the others


What is $3\cdot 7$?

    A) 0.42857142857142855
    B) -4
  * C) 21
    D) 10
    E) None of the others


Now put this into a function:

In [23]:
def my_question_printer(question_list):
    'take a list of the form (q,a1,a2,a3,a4) and pretty-print it'
    formatted_questions = [f'{q}\n\n{mca(answers,answers[0])}' 
                               for q, *answers in question_list]
    for fq in formatted_questions[:2]:
        print(fq,end='\n\n\n')

my_question_printer(question_1_bank)
What is $4\cdot 10$?

    A) 0.4
    B) 14
  * C) 40
    D) -6
    E) None of the others


What is $3\cdot 7$?

    A) -4
    B) 10
  * C) 21
    D) 0.42857142857142855
    E) None of the others


Decorators

Consider again this code, and imagin we are working on it, debugging it.

def quiz_1_elementary_products():
    for _ in range(10):
        a, b = randint(1,9), randint(2,10)
        yield rf'What is ${a}\cdot {b}$?', f'{a*b}', f'{a+b}', f'{a-b}', f'{a/b}'
In [25]:
@with_preview
def quiz_1_elementary_products():
    for _ in range(10):
        a, b = randint(1,9), randint(2,10)
        yield rf'What is ${a}\cdot {b}$?', f'{a*b}', f'{a+b}', f'{a-b}', f'{a/b}'
What is $4\cdot 10$?

    A) -6
    B) 0.4
  * C) 40
    D) 14
    E) None of the others


What is $5\cdot 9$?

    A) -4
  * B) 45
    C) 0.5555555555555556
    D) 14
    E) None of the others


@with_preview is called a decorator. How does it work you ask?

In [26]:
def with_preview(f):
    '"decorate" the generator f by pretty-printing a few of its elements'
    example_of_our_generator = f()
    import itertools
    my_question_printer(itertools.islice(example_of_our_generator,2))
    return f

That's (almost) all folks. Let's recap.

  • Static assessment generators are very powerful. Lists are a universal intermediate format.
  • Write generators, e.g., to produce lists of question-answer pairs, question/multiple choice tuples, etc.
  • Write code that can transform such lists into the output formats you need, e.g., for uploading questions in bulk, to feed into paper assessment generators (e.g., LaTeX exams package, MATHxxxx)
  • Create your own decorators that

    • preview the question you are working on
    • call the output code for you

Thank you for your attention!

Some references you may find useful: