Susam's Python Pages

Mark V. Shaney Junior 0.2.0

2025-12-14T00:00:00Z

Mark V. Shaney Junior 0.2.0 is the second release of this little Markov gibberish generator. This release brings two small changes. First, it now reads the training data from standard input instead of a hardcoded file. Second, the program filename has been changed from mvs.py to mvs to reflect that it is an executable file and can be run as ./mvs on most Unix and Linux systems.

The source and a detailed documentation for this project are available at github.com/susam/mvs.

See also this related blog post and a discussion on Hacker News about it.

Read on website | #python | #programming | #technology

Fed 24 Years of My Blog Posts to a Markov Model

2025-12-13T00:00:00Z

Yesterday I shared a little program called Mark V. Shaney Junior at github.com/susam/mvs. It is a minimal implementation of a Markov text generator inspired by the legendary Mark V. Shaney program from the 1980s. Mark V. Shaney was a synthetic Usenet user that posted messages to various newsgroups using text generated by a Markov model. See the Wikipedia article Mark V. Shaney for more details about it. In this post, I will discuss my implementation of the model, explain how it works and share some of the results produced by it.

Recreational Programming
Gibberish
The Markov Property
Some More Gibberish

Recreational Programming

The program I shared yesterday has only about 30 lines of Python and favours simplicity over efficiency. Even if you have never worked with Markov models before, as long as you know some Python programming, I am quite confident that it will take you less than 20 minutes to understand the whole program and make complete sense of it. I also offer an explanation further below in this post.

As a hobby, I often engage in exploratory programming where I write computer programs not to solve a specific problem but simply to explore a particular idea or topic for the sole purpose of recreation. I must have written small programs to explore Markov chains for various kinds of state spaces over a dozen times by now. Every time, I just pick my last experimental code and edit it to encode the new state space I am exploring. That's usually my general approach to such one-off programs. I have hundreds of tiny little experimental programs lying on my disk at any given time.

Once in a while, I get the itch to take one of those exploratory programs, give it some finishing touches, wrap it up in a nice Git repo along with a README.md, CHANGES.md and the whole shebang and share it on github.com/susam and codeberg.org/susam. The Mark V. Shaney Junior program that I shared yesterday happened to be one such exercise.

Gibberish

If you scroll down the README of this project, you'll find some nice examples of the gibberish produced by this program. The first few examples there are the result of training the model on A Christmas Carol by Charles Dickens, one of my favourite authors. It is often said that Dickens never used fewer words when more would suffice. So I thought there couldn't be a better piece of text when it comes to testing out my tiny Markov model. I'll not reproduce the generated text examples here for the sake of brevity. If you are interested to take a look, just head over to the Gibberish section of the README.

Soon after sharing the project, I wondered what kind of gibberish it would produce if I fed all 24 years of my blog posts and pages into the program. Well, here's one of the results:

$ ./mvs < susam.txt
while a query replace operation is approved by the user. The above
variable defines the build job. It can be incredibly useful while
working on assembly language and machine code. In fact, all internal
resources like the result to refine the search prompt changes from
bck-i-search: to fwd-i-search:. Now type C-SPC (i.e. ctrl+space) to
set a mark causes Emacs to use 32-bit registers like EBP, ESP,
etc. Thus the behaviour is undefined. Such code may behave differently
when compiled with the readily available GNU tools like the shape
of 8. Flipping "P" horizontally makes it a proper quine: cat $0

This is the text that comes out after the program consumes over 200 posts consisting of about 200,000 words. My blog also has a comments section with over 500 comments consisting of about 40,000 words. All comments were excluded while training the model. Here is another output example:

$ ./mvs < susam.txt
enjoy asking "what happens if" and then type M-x zap-up-to-char RET
b. The buffer for this specific video, the actual fare for 8.3 km and
11 are all written from scratch. No prior knowledge is expected to
slow down in future. For now, I will add a statement like x =
0.999..., the inner corner square as discussed in the code segment
into the REPL window. Unlike Slimv, Vlime can work with and debug
executable files, it can be more convenient. M-x: Execute Extended
Command The key sequence M-q invokes the command cat and type TAB to
indent the current

Here is a particularly incoherent but amusing one:

$ ./mvs < susam.txt
Then open a new Lisp source file and the exact answer could harm
students' self-esteem. Scientists have arbitrarily assumed that an
integral domain. However, the string and comment text. To demonstrate
how a build job can trigger itself, pass input to standard output or
standard error), Eshell automatically runs the following command in
Vim and Emacs will copy the message length limit of 512 characters,
etc. For example, while learning to play the game between normal mode
to move the point is on an old dictionary lying around our house and
that is moving to the small and supportive community

No, I have never said anywhere that opening a Lisp source file could harm anyone's self-esteem. The text generator has picked up the 'Lisp source file' phrase from my Lisp in Vim post and the 'self-esteem' bit from the From Perl to Pi post.

The Markov Property

By default, this program looks at trigrams (all sequences of three adjacent words) and creates a map where the first two words of the trigram are inserted as the key and the third word is appended to its list value. This map is the model. In this way, the model captures each pair of adjacent words along with the words that immediately follow each pair. The text generator first chooses a key (a pair of words) at random and selects a word that follows. If there are multiple followers, it picks one uniformly at random. It then repeats this process with the most recent pair of words, consisting of one word from the previous pair and the word that was just picked. It continues to do this until it can no longer find a follower or a fixed word limit (100 by default) is reached. That is pretty much the whole algorithm. There isn't much more to it. It is as simple as it gets. For that reason, I often describe a simple Markov model like this as the 'hello, world' for language models.

If the same trigram occurs multiple times in the training data, the model records the follower word (the third word) multiple times in the list associated with the key (the first two words). This representation can be optimised, of course, by keeping frequencies of the follower words rather than duplicating them in the list, but that is left as an exercise to the reader. In any case, when the text generator chooses a follower for a given pair of words, a follower that occurs more frequently after that pair has a higher probability of being chosen. In effect, the next word is sampled based only on the previous two words and not on the full history of the generated text. This memoryless dependence on the current state is what makes the generator Markov. Formally, for a discrete-time stochastic process, the Markov property can be expressed as \[ P(X_{n+1} \mid X_n, X_{n-1}, \ldots, X_1) = P(X_{n+1} \mid X_n). \] where \( X_n \) represents the \( n \)th state. In our case, each state \( X_n \) is a pair of words \( (w_{n-1}, w_{n}) \) but the state space could just as well consist of other objects, such as a pair of characters, pixel values or musical notes. The sequence of states \( (X_1, X_2, \dots) \) visited by the program forms a Markov chain. The left-hand side of the equation denotes the conditional distribution of the next state \( X_{n+1} \) given the entire history of states \( X_1, X_2, \dots, X_n, \) while the right-hand side conditions only on the current state \( X_n. \) When both are equal, it means that the probability of the next state depends only on the current state and not on the earlier states. This is the Markov property. It applies to the text generation process only, not the training data. The training data is used only to estimate the transition probabilities of the model.

Some More Gibberish

In 2025, given the overwhelming popularity of large language models (LLMs), Markov models like this look unimpressive. Unlike LLMs, a simple Markov model cannot capture global structure or long-range dependencies within the text. It relies entirely on local word transition statistics. Also, these days, one hardly needs a Markov model to generate gibberish; social media provides an ample supply. Nevertheless, I think the simplicity of its design and implementation serves as a good entry point into language models.

In my implementation, the number of words in the key of the map can be set via command line arguments. By default, it is 2 as described above. This value is also known as the order of the model. So by default the order is 2. If we increase it to, say, 3 or 4, the generated text becomes a little more coherent. Here is one such example:

$ ./mvs 4 < susam.txt
It is also possible to search for channels by channel names. For
example, on Libera Chat, to search for all channels with 'python' in
its name, enter the IRC command: /msg alis list python. Although I
have used Libera Chat in the examples above, there are plenty of
infinite fields, so they must all be integral domains too. Consider
the field of rational numbers Q. Another quick way to arrive at this
fact is to observe that when one knight is placed on a type D square,
only two positions for the second knight such that the two knights
attack

Except for a couple of abrupt and meaningless transitions, the text is mostly coherent. We need to be careful about not increasing the order too much. In fact, if we increase the order of the model to 5, the generated text becomes very dry and factual because it begins to quote large portions of the blog posts verbatim. Not much fun can be had with that.

Before I end this post, let me present one final example where I ask it to generate text from an initial prompt:

$ ./mvs 2 100 'Finally we' < susam.txt
Finally we divide this number by a feed aggregrator for Emacs-related
blogs. The following complete key sequences describe the effects of
previous evaluations shall have taken a simple and small to contain
bad content. This provides an interactive byte-compiled Lisp function
in MATLAB and GNU bash 5.1.4 on Debian is easily reproducible in
Windows XP. Older versions might be able to run that server for me it
played a significant burden on me as soon as possible. C-u F: Visit
the marked files or directories in the sense that it was already
initiated and we were to complete the proof.

Apparently, this is how I would sound if I ever took up speaking gibberish!

Read on website | #python | #programming | #technology

Mark V. Shaney Junior 0.1.0

2025-12-12T00:00:00Z

Mark V. Shaney Junior is a minimal implementation of a Markov gibberish generator inspired by the legendary Mark V. Shaney program from the 1980s. Mark V. Shaney was a synthetic Usenet user in the 1980s that posted messages to newsgroups using text generated by a Markov chain program. See the Wikipedia article Mark V. Shaney for more details.

This release introduces the program mvs.py that implements a similar Markov text generator. It reads a text corpus of text from standard input, build an internal Markov model and then generate text using the model.

Please visit github.com/susam/mvs for the source code and some output examples.

Read on website | #python | #programming | #technology

Fizz Buzz with Cosines

2025-11-20T00:00:00Z

Fizz Buzz is a counting game that has become oddly popular in the world of computer programming as a simple test of basic programming skills. The rules of the game are straightforward. Players say the numbers aloud in order beginning with one. Whenever a number is divisible by 3, they say 'Fizz' instead. If it is divisible by 5, they say 'Buzz'. If it is divisible by both 3 and 5, the player says both 'Fizz' and 'Buzz'. Here is a typical Python program that prints this sequence:

for n in range(1, 101):
    if n % 15 == 0:
        print('FizzBuzz')
    elif n % 3 == 0:
        print('Fizz')
    elif n % 5 == 0:
        print('Buzz')
    else:
        print(n)

Here is the output: fizz-buzz.txt. Can we make the program more complicated? The words 'Fizz', 'Buzz' and 'FizzBuzz' repeat in a periodic manner throughout the sequence. What else is periodic? Trigonometric functions! Perhaps we can use trigonometric functions to encode all four rules of the sequence in a single closed-form expression. That is what we are going to explore in this article, for fun and no profit.

By the end, we will obtain a discrete Fourier series that can take any integer \( n \) and select the corresponding text to be printed. In fact, we will derive it using two different methods. First, we will follow a long-winded but hopefully enjoyable approach that relies on a basic understanding of complex exponentiation, geometric series and trigonometric functions. Then, we will obtain the same result through a direct application of the discrete Fourier transform.

Definitions
From Indicator Functions to Cosines
Discrete Fourier Transform
Conclusion

Definitions

Before going any further, we establish a precise mathematical definition for the Fizz Buzz sequence. We begin by introducing a few functions that will help us define the Fizz Buzz sequence later.

Symbol Functions

We define a set of four functions \( \{ s_0, s_1, s_2, s_3 \} \) for integers \( n \) by: \begin{align*} s_0(n) &= n, \\ s_1(n) &= \mathtt{Fizz}, \\ s_2(n) &= \mathtt{Buzz}, \\ s_3(n) &= \mathtt{FizzBuzz}. \end{align*} We call these the symbol functions because they produce every term that appears in the Fizz Buzz sequence. The symbol function \( s_0 \) returns \( n \) itself. The functions \( s_1, \) \( s_2 \) and \( s_3 \) are constant functions that always return the literal words \( \mathtt{Fizz}, \) \( \mathtt{Buzz} \) and \( \mathtt{FizzBuzz} \) respectively, no matter what the value of \( n \) is.

Index Function

We define a function \( f(n) \) for integer \( n \) by \[ f(n) = \begin{cases} 1 & \text{if } 3 \mid n \text{ and } 5 \nmid n, \\ 2 & \text{if } 3 \nmid n \text{ and } 5 \mid n, \\ 3 & \text{if } 3 \mid n \text{ and } 5 \mid n, \\ 0 & \text{otherwise}. \end{cases} \] The notation \( m \mid n \) means that the integer \( m \) divides the integer \( n, \) i.e. \( n \) is a multiple of \( m. \) Equivalently, there exists an integer \( c \) such that \( n = cm . \) Similarly, \( m \nmid n \) means that \( m \) does not divide \( n, \) i.e. \( n \) is not a multiple of \( m. \)

This function covers all four conditions involved in choosing the \( n \)th item of the Fizz Buzz sequence. As we will soon see, this function tells us which of the four symbol functions produces the \( n \)th item of the Fizz Buzz sequence. For this reason, we call \( f(n) \) the index function.

Fizz Buzz Sequence

We now define the Fizz Buzz sequence as the sequence \[ (s_{f(n)}(n))_{n = 1}^{\infty} \] We can expand the first few terms of the sequence explicitly as follows: \begin{align*} (s_{f(n)}(n))_{n = 1}^{\infty} &= (s_{f(1)}(1), \; s_{f(2)}(2), \; s_{f(3)}(3), \; s_{f(4)}(4), \; s_{f(5)}(5), \; s_{f(6)}(6), \; s_{f(7)}(7), \; \dots) \\ &= (s_0(1), \; s_0(2), \; s_1(3), \; s_0(4), s_2(5), \; s_1(6), \; s_0(7), \; \dots) \\ &= (1, \; 2, \; \mathtt{Fizz}, \; 4, \; \mathtt{Buzz}, \; \mathtt{Fizz}, \; 7, \; \dots). \end{align*} Note how the function \( f(n) \) produces an index \( i \) which we then use to select the symbol function \( s_i(n) \) to produce the \( n \)th term of the sequence. This is precisely why we decided to call \( f(n) \) the index function while defining it in the previous section.

From Indicator Functions to Cosines

Here we discuss the first method of deriving our closed form expression, starting with indicator functions and rewriting them using complex exponentials and cosines.

Indicator Functions

Here is the index function \( f(n) \) from the previous section with its cases and conditions rearranged to make it easier to spot interesting patterns: \[ f(n) = \begin{cases} 0 & \text{if } 5 \nmid n \text{ and } 3 \nmid n, \\ 1 & \text{if } 5 \nmid n \text{ and } 3 \mid n, \\ 2 & \text{if } 5 \mid n \text{ and } 3 \nmid n, \\ 3 & \text{if } 5 \mid n \text{ and } 3 \mid n. \end{cases} \] This function helps us select another function \( s_{f(n)}(n) \) which in turn determines the \( n \)th term of the Fizz Buzz sequence. Our goal now is to replace this piecewise formula with a single closed-form expression. To do so, we first define indicator functions \( I_m(n) \) as follows: \[ I_m(n) = \begin{cases} 1 & \text{if } m \mid n, \\ 0 & \text{if } m \nmid n. \end{cases} \] The formula for \( f(n) \) can now be written as: \[ f(n) = \begin{cases} 0 & \text{if } I_5(n) = 0 \text{ and } I_3(n) = 0, \\ 1 & \text{if } I_5(n) = 0 \text{ and } I_3(n) = 1, \\ 2 & \text{if } I_5(n) = 1 \text{ and } I_3(n) = 0, \\ 3 & \text{if } I_5(n) = 1 \text{ and } I_3(n) = 1. \end{cases} \] Do you see a pattern? Here is the same function written as a table:

\( I_5(n) \)	\( I_3(n) \)	\( f(n) \)
\( 0 \)	\( 0 \)	\( 0 \)
\( 0 \)	\( 1 \)	\( 1 \)
\( 1 \)	\( 0 \)	\( 2 \)
\( 1 \)	\( 1 \)	\( 3 \)

\( I_5(n) \)

\( I_3(n) \)

\( f(n) \)

\( 0 \)

\( 1 \)

\( 0 \)

\( 2 \)

\( 1 \)

\( 3 \)

Do you see it now? If we treat the values in the first two columns as binary digits and the values in the third column as decimal numbers, then in each row the first two columns give the binary representation of the number in the third column. For example, \( 3_{10} = 11_2 \) and indeed in the last row of the table, we see the bits \( 1 \) and \( 1 \) in the first two columns and the number \( 3 \) in the last column. In other words, writing the binary digits \( I_5(n) \) and \( I_3(n) \) side by side gives us the binary representation of \( f(n). \) Therefore \[ f(n) = 2 \, I_5(n) + I_3(n). \] We can now write a small program to demonstrate this formula:

for n in range(1, 101):
    s = [n, 'Fizz', 'Buzz', 'FizzBuzz']
    i = (n % 3 == 0) + 2 * (n % 5 == 0)
    print(s[i])

We can make it even shorter at the cost of some clarity:

for n in range(1, 101):
    print([n, 'Fizz', 'Buzz', 'FizzBuzz'][(n % 3 == 0) + 2 * (n % 5 == 0)])

What we have obtained so far is pretty good. While there is no universal definition of a closed-form expression, I think most people would agree that the indicator functions as defined above are simple enough to be permitted in a closed-form expression.

Complex Exponentials

In the previous section, we obtained the formula \[ f(n) = I_3(n) + 2 \, I_5(n) \] which we then used as an index to look up the text to be printed. We also argued that this is a pretty good closed-form expression already.

However, in the interest of making things more complicated, we must ask ourselves: What if we are not allowed to use the indicator functions? What if we must adhere to the commonly accepted meaning of a closed-form expression which allows only finite combinations of basic operations such as addition, subtraction, multiplication, division, integer exponents and roots with integer index as well as functions such as exponentials, logarithms and trigonometric functions. It turns out that the above formula can be rewritten using only addition, multiplication, division and the cosine function. Let us begin the translation. Consider the sum \[ S_m(n) = \sum_{k = 0}^{m - 1} e^{i 2 \pi k n / m}, \] where \( i \) is the imaginary unit and \( n \) and \( m \) are integers. This is a geometric series in the complex plane with ratio \( r = e^{i 2 \pi n / m}. \) If \( n \) is a multiple of \( m , \) then \( n = cm \) for some integer \( c \) and we get \[ r = e^{i 2 \pi n / m} = e^{i 2 \pi c} = 1. \] Therefore, when \( n \) is a multiple of \( m, \) we get \[ S_m(n) = \sum_{k = 0}^{m - 1} e^{i 2 \pi k n / m} = \sum_{k = 0}^{m - 1} 1^k = m. \] If \( n \) is not a multiple of \( m, \) then \( r \ne 1 \) and the geometric series becomes \[ S_m(n) = \frac{r^m - 1}{r - 1} = \frac{e^{i 2 \pi n} - 1}{e^{i 2 \pi n / m} - 1} = 0. \] Therefore, \[ S_m(n) = \begin{cases} m & \text{if } m \mid n, \\ 0 & \text{if } m \nmid n. \end{cases} \] Dividing both sides by \( m, \) we get \[ \frac{S_m(n)}{m} = \begin{cases} 1 & \text{if } m \mid n, \\ 0 & \text{if } m \nmid n. \end{cases} \] But the right-hand side is \( I_m(n). \) Therefore \[ I_m(n) = \frac{S_m(n)}{m} = \frac{1}{m} \sum_{k = 0}^{m - 1} e^{i 2 \pi k n / m}. \]

Cosines

We begin with Euler's formula \[ e^{i x} = \cos x + i \sin x \] where \( x \) is a real number. From this formula, we get \[ e^{i x} + e^{-i x} = 2 \cos x. \] Therefore \begin{align*} I_3(n) &= \frac{1}{3} \sum_{k = 0}^2 e^{i 2 \pi k n / 3} \\ &= \frac{1}{3} \left( 1 + e^{i 2 \pi n / 3} + e^{i 4 \pi n / 3} \right) \\ &= \frac{1}{3} \left( 1 + e^{i 2 \pi n / 3} + e^{-i 2 \pi n / 3} \right) \\ &= \frac{1}{3} + \frac{2}{3} \cos \left( \frac{2 \pi n}{3} \right). \end{align*} The third equality above follows from the fact that \( e^{i 4 \pi n / 3} = e^{i 6 \pi n / 3} e^{-i 2 \pi n / 3} = e^{i 2 \pi n} e^{-i 2 \pi n/3} = e^{-i 2 \pi n / 3} \) when \( n \) is an integer.

The function above is defined for integer values of \( n \) but we can extend its formula to real \( x \) and plot it to observe its shape between integers. As expected, the function takes the value \( 1 \) whenever \( x \) is an integer multiple of \( 3 \) and \( 0 \) whenever \( x \) is an integer not divisible by \( 3. \)

Graph of \( \frac{1}{3} + \frac{2}{3} \cos \left( \frac{2 \pi x}{3} \right) \)

Similarly, \begin{align*} I_5(n) &= \frac{1}{5} \sum_{k = 0}^4 e^{i 2 \pi k n / 5} \\ &= \frac{1}{5} \left( 1 + e^{i 2 \pi n / 5} + e^{i 4 \pi n / 5} + e^{i 6 \pi n / 5} + e^{i 8 \pi n / 5} \right) \\ &= \frac{1}{5} \left( 1 + e^{i 2 \pi n / 5} + e^{i 4 \pi n / 5} + e^{-i 4 \pi n / 5} + e^{-i 2 \pi n / 5} \right) \\ &= \frac{1}{5} + \frac{2}{5} \cos \left( \frac{2 \pi n}{5} \right) + \frac{2}{5} \cos \left( \frac{4 \pi n}{5} \right). \end{align*} Extending this expression to real values of \( x \) allows us to plot its shape as well. Once again, the function takes the value \( 1 \) at integer multiples of \( 5 \) and \( 0 \) at integers not divisible by \( 5. \)

Graph of \( \frac{1}{5} + \frac{2}{5} \cos \left( \frac{2 \pi x}{5} \right) + \frac{2}{5} \cos \left( \frac{4 \pi x}{5} \right) \)

Recall that we expressed \( f(n) \) as \[ f(n) = I_3(n) + 2 \, I_5(n). \] Substituting these trigonometric expressions yields \[ f(n) = \frac{1}{3} + \frac{2}{3} \cos \left( \frac{2 \pi n}{3} \right) + 2 \cdot \left( \frac{1}{5} + \frac{2}{5} \cos \left( \frac{2 \pi n}{5} \right) + \frac{2}{5} \cos \left( \frac{4 \pi n}{5} \right) \right). \] A straightforward simplification gives \[ f(n) = \frac{11}{15} + \frac{2}{3} \cos \left( \frac{2 \pi n}{3} \right) + \frac{4}{5} \cos \left( \frac{2 \pi n}{5} \right) + \frac{4}{5} \cos \left( \frac{4 \pi n}{5} \right). \] We can extend this expression to real \( x \) and plot it as well. The resulting curve takes the values \( 0, 1, 2 \) and \( 3 \) at integer points, as desired.

Graph of \( \frac{11}{15} + \frac{2}{3} \cos \left( \frac{2 \pi x}{3} \right) + \frac{4}{5} \cos \left( \frac{2 \pi x}{5} \right) + \frac{4}{5} \cos \left( \frac{4 \pi x}{5} \right) \)

Now we can write our Python program as follows:

from math import cos, pi
for n in range(1, 101):
    s = [n, 'Fizz', 'Buzz', 'FizzBuzz']
    i = round(11 / 15 + (2 / 3) * cos(2 * pi * n / 3)
                      + (4 / 5) * cos(2 * pi * n / 5)
                      + (4 / 5) * cos(4 * pi * n / 5))
    print(s[i])

Discrete Fourier Transform

The keen-eyed might notice that the expression we obtained for \( f(n) \) is a discrete Fourier series. This is not surprising, since the output of a Fizz Buzz program depends only on \( n \bmod 15. \) Any function on a finite cyclic group can be written exactly as a finite Fourier expansion. In this section, we obtain \( f(n) \) using the discrete Fourier transform. It is worth mentioning that the calculations presented here are quite tedious to do by hand. Nevertheless, this section offers a glimpse of how such calculations are performed. By the end, we will arrive at exactly the same \( f(n) \) as before. There is nothing new to discover here. We simply obtain the same result by a more direct but more laborious method. If this doesn't sound interesting, you may safely skip the subsections that follow.

One Period of Fizz Buzz

We know that \( f(n) \) is a periodic function with period \( 15. \) To apply the discrete Fourier transform, we look at one complete period of the function using the values \( n = 0, 1, \dots, 14. \) Over this period, we have: \begin{array}{c|ccccccccccccccc} n & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 & 10 & 11 & 12 & 13 & 14 \\ \hline f(n) & 3 & 0 & 0 & 1 & 0 & 2 & 1 & 0 & 0 & 1 & 2 & 0 & 1 & 0 & 0 \end{array} The discrete Fourier series of \( f(n) \) is \[ f(n) = \sum_{k = 0}^{14} c_k \, e^{i 2 \pi k n / 15} \] where the Fourier coefficients \( c_k \) are given by the discrete Fourier transform \[ c_k = \frac{1}{15} \sum_{n = 0}^{14} f(n) e^{-i 2 \pi k n / 15}. \] for \( k = 0, 1, \dots, 14. \) The formula for \( c_k \) is called the discrete Fourier transform (DFT). The formula for \( f(n) \) is called the inverse discrete Fourier transform (IDFT).

Fourier Coefficients

Let \( \omega = e^{-i 2 \pi / 15}. \) Then using the values of \( f(n) \) from the table above, the DFT becomes: \[ c_k = \frac{3 + \omega^{3k} + 2 \omega^{5k} + \omega^{6k} + \omega^{9k} + 2 \omega^{10k} + \omega^{12k}}{15}. \] Substituting \( k = 0, 1, 2, \dots, 14 \) into the above equation gives us the following Fourier coefficients: \begin{align*} c_{0} &= \frac{11}{15}, \\ c_{3} &= c_{6} = c_{9} = c_{12} = \frac{2}{5}, \\ c_{5} &= c_{10} = \frac{1}{3}, \\ c_{1} &= c_{2} = c_{4} = c_{7} = c_{8} = c_{11} = c_{13} = c_{14} = 0. \end{align*} Calculating these Fourier coefficients by hand can be rather tedious. In practice they are almost always calculated using numerical software, computer algebra systems or even simple code such as the example here: fizz-buzz-fourier.py.

Inverse Transform

Once the coefficients are known, we can substitute them into the inverse transform introduced earlier to obtain \begin{align*} f(n) &= \sum_{k = 0}^{14} c_k \, e^{i 2 \pi k n / 15} \\[1.5em] &= \frac{11}{15} + \frac{2}{5} \left( e^{i 2 \pi \cdot 3n / 15} + e^{i 2 \pi \cdot 6n / 15} + e^{i 2 \pi \cdot 9n / 15} + e^{i 2 \pi \cdot 12n / 15} \right) \\ & \phantom{=\frac{11}{15}} + \frac{1}{3} \left( e^{i 2 \pi \cdot 5n / 15} + e^{i 2 \pi \cdot 10n / 15} \right) \\[1em] &= \frac{11}{15} + \frac{2}{5} \left( e^{i 2 \pi \cdot 3n / 15} + e^{i 2 \pi \cdot 6n / 15} + e^{-i 2 \pi \cdot 6n / 15} + e^{-i 2 \pi \cdot 3n / 15} \right) \\ & \phantom{=\frac{11}{15}} + \frac{1}{3} \left( e^{i 2 \pi \cdot 5n / 15} + e^{-i 2 \pi \cdot 5n / 15} \right) \\[1em] &= \frac{11}{15} + \frac{2}{5} \left( 2 \cos \left( \frac{2 \pi n}{5} \right) + 2 \cos \left( \frac{4 \pi n}{5} \right) \right) \\ & \phantom{=\frac{11}{15}} + \frac{1}{3} \left( 2 \cos \left( \frac{2 \pi n}{3} \right) \right) \\[1em] &= \frac{11}{15} + \frac{4}{5} \cos \left( \frac{2 \pi n}{5} \right) + \frac{4}{5} \cos \left( \frac{4 \pi n}{5} \right) + \frac{2}{3} \cos \left( \frac{2 \pi n}{3} \right). \end{align*} This is exactly the same expression for \( f(n) \) we obtained in the previous section. We see that the Fizz Buzz index function \( f(n) \) can be expressed precisely using the machinery of Fourier analysis.

Conclusion

To summarise, we have defined the Fizz Buzz sequence as \[ (s_{f(n)}(n))_{n = 1}^{\infty} \] where \[ f(n) = \frac{11}{15} + \frac{2}{3} \cos \left( \frac{2 \pi n}{3} \right) + \frac{4}{5} \cos \left( \frac{2 \pi n}{5} \right) + \frac{4}{5} \cos \left( \frac{4 \pi n}{5} \right). \] and \( s_0(n) = n, \) \( s_1(n) = \mathtt{Fizz}, \) \( s_2(n) = \mathtt{Buzz} \) and \( s_3(n) = \mathtt{FizzBuzz}. \) A Python program to print the Fizz Buzz sequence based on this definition was presented earlier. That program can be written more succinctly as follows:

from math import cos, pi
for n in range(1, 101):
    print([n, 'Fizz', 'Buzz', 'FizzBuzz'][round(11 / 15 + (2 / 3) * cos(2 * pi * n / 3) + (4 / 5) * (cos(2 * pi * n / 5) + cos(4 * pi * n / 5)))])

We can also wrap this up nicely in a shell one-liner, in case you want to share it with your friends and family and surprise them:

python3 -c 'from math import cos, pi; [print([n, "Fizz", "Buzz", "FizzBuzz"][round(11/15 + (2/3) * cos(2*pi*n/3) + (4/5) * (cos(2*pi*n/5) + cos(4*pi*n/5)))]) for n in range(1, 101)]'

We have taken a simple counting game and turned it into a trigonometric construction consisting of a discrete Fourier series with three cosine terms and four coefficients. None of this makes Fizz Buzz any easier. Quite the contrary. But it does show that every \( \mathtt{Fizz} \) and \( \mathtt{Buzz} \) now owes its existence to a particular set of Fourier coefficients. We began with the modest goal of making this simple problem more complicated. I think it is safe to say that we did not fall short.

Elliptical Python Programming

2025-04-10T00:00:00Z

One thing I love about Python is how it comes with its very own built-in zen. In moments of tribulations, when I am wrestling with crooked code and tangled thoughts, I often find solace in its timeless wisdom. Here's a glimpse of the clarity it provides:

$ python3 -m this | grep e-
There should be one-- and preferably only one --obvious way to do it.

Indeed, there is one and only one obvious way to write the number 1 in Python, like so:

>>> --(...==...)
1

You may, quite naturally, place several ones adjacently to produce larger integers:

>>> --(...==...)--(...==...)
2

And so on ad infinitum or until your heap collapses like a poorly made soufflé. Now, the 'pre-decrement operator' at the beginning is entirely optional, much like the plus sign when you write '+5 biscuits' in a letter to your grandmother. It's not wrong, but it is unnecessary. So unless you want to look peculiar to your colleagues, you would likely want to adopt a more conventional style, such as this:

>>> (...==...)--(...==...)--(...==...)
3

Now, all computer programs are, in some sense, just a long, earnest stream of bits. It is currently fashionable to bundle these bits into groups of eight and write them as integers. Following this trend, we can compute absolutely anything that is computable as long as we know exactly what integers to write. Now, I wouldn't want to bore you with the finer details of computer science, not in this day and age, fascinating as they may be. I trust you are quite capable of drawing the rest of the f... well, feathered, nocturnal bird. Once you've grasped the basics, a typical first Python program might look something like this:

exec('%c'*((...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...))%(((...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...)),((...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...))--((...==...)--(...==...)),((...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...)),((...==...)--(...==...)--(...==...))**((...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...))--((...==...)--(...==...)),((...==...)--(...==...)--(...==...)--(...==...))**((...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...))--((...==...)--(...==...)--(...==...)--(...==...)),((...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...)--(...==...)),((...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...))**((...==...)--(...==...))--((...==...)--(...==...)--(...==...)),((...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...)--(...==...))**((...==...)--(...==...))--((...==...)--(...==...)--(...==...)--(...==...)),((...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...)--(...==...))**((...==...)--(...==...))--(...==...),*(((...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...))**((...==...)--(...==...)--(...==...)),)*((...==...)--(...==...)),((...==...)--(...==...)--(...==...))**((...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...))--((...==...)--(...==...)--(...==...)),((...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...))--((...==...)--(...==...)),((...==...)--(...==...))**((...==...)--(...==...)--(...==...)--(...==...)--(...==...)),((...==...)--(...==...)--(...==...)--(...==...))**((...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...))--((...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...)),((...==...)--(...==...)--(...==...))**((...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...))--((...==...)--(...==...)--(...==...)),((...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...))--((...==...)--(...==...)),((...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...))**((...==...)--(...==...)--(...==...)),((...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...)--(...==...))**((...==...)--(...==...)),((...==...)--(...==...)--(...==...)--(...==...)--(...==...)--(...==...))**((...==...)--(...==...))--((...==...)--(...==...)--(...==...)),((...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...))*((...==...)--(...==...)--(...==...)--(...==...)--(...==...))--(...==...)))

Now you might be wondering if this is really the way one ought to write production Python code. Isn't it too much trouble to type those dots over and over again? Not if you remap your tab key to type three dots, of course. But I understand not everyone likes to remap their keys like this. In particular, there exists a peculiar species of mammal known to remap their tab key to parentheses. They claim it leads to enlightenment. Such enlightened living forms may find the following program more convenient to type:

exec('%c'*((()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==())--(()==())--(()==())--(()==()))%(((()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==())--(()==())--(()==())--(()==())),((()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==())--(()==())--(()==())--(()==()))--((()==())--(()==())),((()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==())--(()==())--(()==())--(()==())),((()==())--(()==())--(()==()))**((()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==()))--((()==())--(()==())),((()==())--(()==())--(()==())--(()==()))**((()==())--(()==()))*((()==())--(()==())--(()==())--(()==())--(()==())--(()==())--(()==()))--((()==())--(()==())--(()==())--(()==())),((()==())--(()==()))*((()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==())--(()==())),((()==())--(()==())--(()==())--(()==())--(()==())--(()==()))**((()==())--(()==()))--((()==())--(()==())--(()==())),((()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==())--(()==()))**((()==())--(()==()))--((()==())--(()==())--(()==())--(()==())),((()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==())--(()==()))**((()==())--(()==()))--(()==()),*(((()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==()))**((()==())--(()==())--(()==())),)*((()==())--(()==())),((()==())--(()==())--(()==()))**((()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==()))--((()==())--(()==())--(()==())),((()==())--(()==())--(()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==())--(()==())--(()==())--(()==()))--((()==())--(()==())),((()==())--(()==()))**((()==())--(()==())--(()==())--(()==())--(()==())),((()==())--(()==())--(()==())--(()==()))**((()==())--(()==()))*((()==())--(()==())--(()==())--(()==())--(()==())--(()==())--(()==()))--((()==())--(()==())--(()==())--(()==())--(()==())--(()==())--(()==())),((()==())--(()==())--(()==()))**((()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==()))--((()==())--(()==())--(()==())),((()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==())--(()==())--(()==())--(()==()))--((()==())--(()==())),((()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==()))**((()==())--(()==())--(()==())),((()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==())--(()==()))**((()==())--(()==())),((()==())--(()==())--(()==())--(()==())--(()==())--(()==()))**((()==())--(()==()))--((()==())--(()==())--(()==())),((()==())--(()==()))*((()==())--(()==())--(()==())--(()==()))*((()==())--(()==())--(()==())--(()==())--(()==()))--(()==())))

This program is functionally equivalent to the earlier one. But Python isn't meant for enlightenment. It's meant for getting things done. And to get things done, code should be readable, maintainable and ideally not resemble an ancient summoning ritual. That's why I personally prefer the earlier style, the one with the ellipses. It gracefully avoids the disconcerting void that lurks within the parentheses. After all, programs must be written for people to read and only incidentally for machines to execute.

Finally, I must emphasise that you should never deploy code like this in production. If you plan to write code like this for your production CGI scripts, I implore you to add some ellipses for logging. When dung inevitably collides with the fan, you'll be immensely glad you scattered some useful logs amidst the ellipses that hold together your business logic. With that small piece of unsolicited advice, I'll end this brief distraction from scrolling through endless arguments on Internet forums. Happy coding and may your parentheses stay balanced (and may your ellipses be the punctuation that ...

Read on website | #absurd | #python | #programming | #humour

Negative Lookahead Assertion

2024-11-20T00:00:00Z

Here is an example of negative lookahead assertion in regular expression using Python:

import re
strings = ['foo', 'bar', 'baz', 'foo-bar', 'bar-baz', 'baz-foo']
matches = [s for s in strings if re.search('^(?!.*foo)', s)]
print(matches)

The regular expression ^(?!.*foo) in the above example matches strings that do not contain the pattern foo. The above code example produces the following output:

['bar', 'baz', 'bar-baz']

Of course, it is much simpler to use an ordinary regular expression that matches foo and then invert the result of the match to ignore strings that contain foo. For example, consider the following straightforward solution:

matches = [s for s in strings if not re.search('foo', s)]

This example produces the same result as the earlier example but with less complexity. However, there are situations where, as a user of certain software tool, we might not have control over how the tool applies the regular expression. Some tools only allow us to provide a pattern and then they automatically select strings that match the pattern. In such cases, if we need to select strings that do not match a given pattern, negative lookahead assertions become quite useful, provided the regular expression flavour supported by the tool allows the use of negative lookahead assertions.

Read on website | #python | #programming | #technology

Import Readline

2022-02-24T00:00:00Z

Toy REPL

Let us first write a tiny Python program to create a toy read-eval-print-loop (REPL) that does only one thing: add all integers entered as input into the REPL prompt. Here is the program:

while True:
    try:
        line = input('> ')
        print(sum([int(n) for n in line.split()]))
    except ValueError as e:
        print('error:', e)
    except (KeyboardInterrupt, EOFError):
        break

Here is how it works:

$ python3 repl.py
> 10 20 30
60
> 40 50 60
150
>

If we now type ↑ (the up arrow key) or ctrl+p to bring back the previous input, we see something like the following instead:

> ^[[A^[[A^P^P

It shows the keys typed literally rather than bringing up previous input like most other interactive programs with a command-line interface do. The other programs that do bring up the previous input are able to do so because they provide line editing and history capability, often with the help of a line editing and history library like GNU Readline (libreadline) or BSD Editline (libedit).

Can we have a similar line editing and history capability for our toy REPL? After all, the Python REPL itself offers such a line editing facility. Surely there must be a way to have this facility for our own programs too. Indeed there is!

Line Editing and History

To enable line editing and history in our toy REPL, we just need to add import readline to our program. Here is how our program would look:

import readline

while True:
    try:
        line = input('> ')
        print(sum([int(n) for n in line.split()]))
    except ValueError as e:
        print('error:', e)
    except (KeyboardInterrupt, EOFError):
        break

Now ↑, ctrl+p, etc. work as expected.

$ python3 repl.py
> 10 20 30
60
> 40 50 60
150
> 40 50 60

The last line of input in the example above is obtained by typing either ↑ or ctrl+p. In fact, all of the line editing keys like ctrl+a to go to the beginning of the line, ctrl+k to kill the line after the cursor, etc. work as expected. The exact list of default key-bindings supported depends on the underlying line editing library being used by the readline module. The underlying library may be either the GNU Readline library or the BSD Editline library. There are some minor differences regarding the list of default key-bindings between these two libraries.

History File

What we have done so far achieves the goal of bringing up previous inputs from the history. However, it does not bring back inputs from a previous invocation of the REPL. For example, if we start our toy REPL, enter some inputs, then quit it (say, by typing ctrl+c), start our toy REPL again and type ↑ or ctrl+p, it does not bring back the input from the previous invocation. For a full-blown REPL meant for sophisicated usage, we may want to preserve the history between different invocations of the REPL. This can be achieved by using the read_history_file() and write_history_file() functions as shown below:

import readline
import os

HISTORY_FILE = os.path.expanduser('~/.repl_history')
if os.path.exists(HISTORY_FILE):
    readline.read_history_file(HISTORY_FILE)

while True:
    try:
        line = input('> ')
        readline.write_history_file(HISTORY_FILE)
        print(sum([int(n) for n in line.split()]))
    except ValueError as e:
        print('error:', e)
    except (KeyboardInterrupt, EOFError):
        break

For more information on how to use this module, see the Python readline documentation.

Readline Wrapper

At this point, it is worth mentioning that there are many interactive CLI tools that do not have line editing and history capabilities. They behave like our first toy REPL example in this post. Fortunately, there is the wonderful readline wrapper utility known as rlwrap that can be used to enable line editing and history in such tools. This utility can often be easily installed from package repositories of various operating systems. Here is a demonstration of this tool:

$ rlwrap cat
hello
hello
world
world
world

The last line of input in the example above is obtained by typing either ↑ or ctrl+p. In the above example, the input history is automatically saved to ~/.cat_history, so it is possible to bring back inputs from a previous invocation of the command.

Obligatory Joke

Finally, an obligatory XKCD comic to conclude this post:

Python by Randall Munroe (Source: https://xkcd.com/353/)

While the days of achieving air flight with a single import statement might still be a few decades away, we do have the luxury to enable line editing and history in our REPLs with a single such statement right now.

Read on website | #python | #programming | #technology

Unix Timestamp 1600000000

2020-09-12T00:00:00Z

At 2020-09-13 12:26:40 UTC, the Unix timestamp is going to turn 1600000000.

Unix Timestamp Conversion

The following subsections show a few examples of converting the Unix timestamp to a human-readable date.

Python

$ python3 -q
>>> from datetime import datetime
>>> datetime.utcfromtimestamp(1_600_000_000)
datetime.datetime(2020, 9, 13, 12, 26, 40)

GNU date (Linux)

$ date -ud @1600000000
Sun Sep 13 12:26:40 UTC 2020

BSD date (macOS, FreeBSD, OpenBSD, etc.)

$ date -ur 1600000000
Sun Sep 13 12:26:40 UTC 2020

Other Such Dates

All such dates (in UTC) until the end of the current century:

$ python3 -q
>>> from datetime import datetime
>>> for t in range(0, 4_200_000_000, 100_000_000):
...     print(f'{t:13_d} - {datetime.utcfromtimestamp(t)}')
...
            0 - 1970-01-01 00:00:00
  100_000_000 - 1973-03-03 09:46:40
  200_000_000 - 1976-05-03 19:33:20
  300_000_000 - 1979-07-05 05:20:00
  400_000_000 - 1982-09-04 15:06:40
  500_000_000 - 1985-11-05 00:53:20
  600_000_000 - 1989-01-05 10:40:00
  700_000_000 - 1992-03-07 20:26:40
  800_000_000 - 1995-05-09 06:13:20
  900_000_000 - 1998-07-09 16:00:00
1_000_000_000 - 2001-09-09 01:46:40
1_100_000_000 - 2004-11-09 11:33:20
1_200_000_000 - 2008-01-10 21:20:00
1_300_000_000 - 2011-03-13 07:06:40
1_400_000_000 - 2014-05-13 16:53:20
1_500_000_000 - 2017-07-14 02:40:00
1_600_000_000 - 2020-09-13 12:26:40
1_700_000_000 - 2023-11-14 22:13:20
1_800_000_000 - 2027-01-15 08:00:00
1_900_000_000 - 2030-03-17 17:46:40
2_000_000_000 - 2033-05-18 03:33:20
2_100_000_000 - 2036-07-18 13:20:00
2_200_000_000 - 2039-09-18 23:06:40
2_300_000_000 - 2042-11-19 08:53:20
2_400_000_000 - 2046-01-19 18:40:00
2_500_000_000 - 2049-03-22 04:26:40
2_600_000_000 - 2052-05-22 14:13:20
2_700_000_000 - 2055-07-24 00:00:00
2_800_000_000 - 2058-09-23 09:46:40
2_900_000_000 - 2061-11-23 19:33:20
3_000_000_000 - 2065-01-24 05:20:00
3_100_000_000 - 2068-03-26 15:06:40
3_200_000_000 - 2071-05-28 00:53:20
3_300_000_000 - 2074-07-28 10:40:00
3_400_000_000 - 2077-09-27 20:26:40
3_500_000_000 - 2080-11-28 06:13:20
3_600_000_000 - 2084-01-29 16:00:00
3_700_000_000 - 2087-04-01 01:46:40
3_800_000_000 - 2090-06-01 11:33:20
3_900_000_000 - 2093-08-01 21:20:00
4_000_000_000 - 2096-10-02 07:06:40
4_100_000_000 - 2099-12-03 16:53:20

Update

Here is a screenshot I took at Unix timestamp 1600000000: twitter.com/susam/status/130512093609862758.

Reproduced as text below:

$ date -u; date; date +%s
Sun Sep 13 12:26:39 UTC 2020
Sun Sep 13 17:56:39 IST 2020
1599999999
$ date -u; date; date +%s
Sun Sep 13 12:26:40 UTC 2020
Sun Sep 13 17:56:40 IST 2020
1600000000

An important point worth noting from the POSIX.1-2008 specification:

Coordinated Universal Time (UTC) includes leap seconds. However, in POSIX time (seconds since the Epoch), leap seconds are ignored (not applied) to provide an easy and compatible method of computing time differences. Broken-down POSIX time is therefore not necessarily UTC, despite its appearance.

See § A.4.16 of the POSIX.1-2008 specification for more details.

Peculiar Self-References

2019-02-21T00:00:00Z

Peculiar Results

Here is a tiny Python example that creates a self-referential list and demonstrates the self-reference:

>>> a = a[0] = [0]
>>> a
[[...]]
>>> a[0]
[[...]]
>>> a[0][0]
[[...]]
>>> a is a[0]
True

The output shows that a[0] refers to a itself which makes it a self-referential list. Why does this simple code create a self-referential list? Should it not have failed with NameError because a is not yet defined while assigning the list [0] to a[0]?

Here is another similar example that creates a self-referential list too:

>>> a = a[0] = [0, 0]
>>> a
[[...], 0]

Here is a similar example for dictionary:

>>> a = a[0] = {}
>>> a
{0: {...}}

Note that 0 is used as a dictionary key in the above example. Here is another very simple example that uses a string key:

>>> a = a['k'] = {}
>>> a
{'k': {...}}

The Language Reference

My first guess was that the statement

a = a[0] = [0]

behaves like

new = [0]
a = new
a[0] = new

which would indeed create a self-referential list.

Section 7.2 (Assignment statements) of The Python Language Reference confirms this behaviour. Quoting the relevant part from this section here:

Assignment statements are used to (re)bind names to values and to modify attributes or items of mutable objects:
assignment_stmt ::=  (target_list "=")+ (starred_expression | yield_expression)
target_list     ::=  target ("," target)* [","]
target          ::=  identifier
                     | "(" [target_list] ")"
                     | "[" [target_list] "]"
                     | attributeref
                     | subscription
                     | slicing
                     | "*" target
(See section Primaries for the syntax definitions for attributeref, subscription and slicing.)

An assignment statement evaluates the expression list (remember that this can be a single expression or a comma-separated list, the latter yielding a tuple) and assigns the single resulting object to each of the target lists, from left to right.

We see that the assignment statement is defined as follows:

assignment_stmt ::=  (target_list "=")+ (starred_expression | yield_expression)

Thus the statement

a = a[0] = [0]

has two target_list elements (a and a[0]) and a starred_expression element ([0]). As a result, the same list on the right-hand-side is assigned to both a and a[0], from left to right, i.e. the list [0] is first assigned to a, then a[0] is set to the same list. As a result, a[0] is set to a itself.

The behaviour of the statement

a = a[0] = {}

can be explained in a similar way. The dictionary object on the right-hand-side is first assigned to a. Then a key 0 is inserted within the same dictionary. Finally the value of a[0] is set to the same dictionary. In other words, a[0] is set to a itself.

More Experiments

The evaluation of the expression list on the right hand side first and then assigning the result to each target list from left to right explains the behaviour we observed in the previous sections. This left-to-right assignment is quite uncommon among mainstream programming languages. For example, in C, C++, Java and JavaScript the simple assignment operator (=) has right-to-left associativity. The left-to-right assignment in Python can be further demonstrated with some intentional errors. Here is an example:

>>> a[0] = a = [0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'a' is not defined

In this example, when the assignment to a[0] occurs, the variable named a is not defined yet, so it leads to NameError.

Here is another example:

>>> a = a[0] = 0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object does not support item assignment

In this example, 0 is first assigned to a. Then a[0] needs to be evaluated before 0 can be assigned to it but this evaluation fails because a is an int, a type that does not support subscription (also known as indexing), so it fails with TypeError.

Read on website | #python | #programming | #technology

Fixed Bits of Version 4 UUID

2015-04-10T00:00:00Z

Universally Unique Identifiers or UUIDs are a popular way of creating identifiers that are unique for practical purposes. Quoting from RFC 4122 below:

This specification defines a Uniform Resource Name namespace for UUIDs (Universally Unique IDentifier), also known as GUIDs (Globally Unique IDentifier). A UUID is 128 bits long and requires no central registration process.

These 128-bit identifiers are typically represented as 32 hexadecimal digits, displayed in five groups separated by hyphens. There are various variants and versions of UUIDs which differ in how the identifiers are encoded in binary and how they are generated. In this post, we are going to focus only on variant 1 of version 4 UUIDs, also known simply as version 4 UUIDs or random UUIDs. Here are a couple examples of version 4 UUIDs generated using Python:

>>> import uuid
>>> str(uuid.uuid4())
'980ddc6a-2c56-44da-ac71-9e6bfc924e25'
>>> str(uuid.uuid4())
'10c3fcde-96a0-4c9e-905b-443b00ceeb01'

Version 4 UUID is one of the most popular type of UUIDs in use today. Unlike the other versions, this version does not require external inputs like MAC address, sequence number, current time, etc. All except six bits are generated randomly in version 4 UUIDs. The six non-random bits are fixed. They represent the version and variant of the UUID. Here is a tiny Python program that demonstrates the first set of fixed bits:

while str(uuid.uuid4())[14] == '4': pass

The above program is an infinite loop. So is this:

while str(uuid.uuid4())[19] in ['8', '9', 'a', 'b']: pass

The above infinite loops show that the hexademical digit at index 14 must always be 4. Similarly, the hexadecimal digit at index 19 must always be one of 8, 9, a and b. We can see the two examples of version 4 UUIDs mentioned earlier and confirm that this is indeed the case. Here are a few more examples that illustrate this pattern:

527218be-a09e-4d0e-86ce-c39d1348d953
14163389-2eea-4e30-9124-fcf2451eb9fc
c21b57cc-2a4e-4425-a2f4-129256562599
37700270-6deb-4a73-bbcd-d47c6e20b567

The digit after the second hyphen is at index 14 and indeed this digit is always 4. Similarly, the hexadecimal digit after the third hyphen is at index 19 and indeed it is always one of 8, 9, a and b.

If we number the octets in the identifiers as 0, 1, 2, etc. where 0 represents the most significant octet (the leftmost pair of hexadecimal digits in the string representations above), then with a careful study of section 4.1.1 of RFC 4122 we know that the first two most significant bits of octet 8 represent the variant number. Since we are working with variant 1 of version 4 UUIDs, these two bits must be 1 and 0. As a result, octet 8 must be of the form 10xx xxxx in binary where each x represents an independent random bit. Thus, in binary, the four most significant bits of octet 8 must be one of 1000, 1001, 1010 and 1011. This explains why we always see the hexadecimal digit 8, 9, a or b at this position.

Similarly, a study of section 4.1.2 and section 4.1.3 of the RFC shows that the four most significant bits of octet 6 must be set to 0100 to represent the version number 4. This explains why we always see the hexadecimal digit 4 here.

Section 4.4 of RFC 4122 further summarises these points. To summarise, version 4 UUIDs, although 128 bits in length, have 122 bits of randomness. They have six fixed bits that represent its version and variant.

Read on website | #python | #programming | #technology

Zero Length Regular Expression

2010-05-03T00:00:00Z

This post presents a list of how zero length regular expression is handled in various tools and programming languages. All of them compile the zero length regular expression pattern and the regular expression matches all strings.

GNU grep

$ printf "foo\nbar\n" | grep ""
foo
bar

BSD grep

$ printf "foo\nbar\n" | grep ""
foo
bar

Perl

$ perl -e 'print(("foo" =~ //) . "\n")'
1

Python

$ python
Python 2.5.2 (r252:60911, Jan  4 2009, 21:59:32)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import re; re.compile('').search('foo')
<_sre.SRE_Match object at 0x7fc6c5a2c510>

Java

$ cat RegexExperiment.java
import java.util.regex.Pattern;
import java.util.regex.Matcher;

public class RegexExperiment
{
    public static void main(String[] args)
    {
        System.out.println(Pattern.compile("").matcher("foo").find());
    }
}
$ javac RegexExperiment.java && java RegexExperiment
true

MzScheme

$ mzscheme
Welcome to MzScheme v4.0.1 [3m], Copyright (c) 2004-2008 PLT Scheme Inc.
> (regexp-match "" "foo")
("")

CLISP

$ clisp
  i i i i i i i       ooooo    o        ooooooo   ooooo   ooooo
  I I I I I I I      8     8   8           8     8     o  8    8
  I  \ `+' /  I      8         8           8     8        8    8
   \  `-+-'  /       8         8           8      ooooo   8oooo
    `-__|__-'        8         8           8           8  8
        |            8     o   8           8     o     8  8
  ------+------       ooooo    8oooooo  ooo8ooo   ooooo   8

Welcome to GNU CLISP 2.44.1 (2008-02-23) <http://clisp.cons.org/>

Copyright (c) Bruno Haible, Michael Stoll 1992, 1993
Copyright (c) Bruno Haible, Marcus Daniels 1994-1997
Copyright (c) Bruno Haible, Pierpaolo Bernardi, Sam Steingold 1998
Copyright (c) Bruno Haible, Sam Steingold 1999-2000
Copyright (c) Sam Steingold, Bruno Haible 2001-2008

Type :h and hit Enter for context help.

[1]> (regexp:match "" "foo")
#S(REGEXP:MATCH :START 0 :END 0)

C

$ ls -l /usr/lib/libpcre.so*
lrwxrwxrwx 1 root root     17 May  3 15:15 /usr/lib/libpcre.so -> libpcre.so.3.12.1
lrwxrwxrwx 1 root root     17 Jan  6 14:57 /usr/lib/libpcre.so.3 -> libpcre.so.3.12.1
-rw-r--r-- 1 root root 162816 Jul 14  2008 /usr/lib/libpcre.so.3.12.1
susam@swift:~$ cat pcre.c
#include <stdio.h>
#include <string.h>
#include <pcre.h>

#include <stdio.h>
#include <string.h>
#include <pcre.h>

int main(int argc, char **argv)
{
    pcre *p;
    char *re = "";
    char *s  = "foo";
    const char *errmsg;
    int errpos;
    int ovector[10];
    int ret;

    p = pcre_compile(re, 0, &errmsg, &errpos, NULL);
    ret = pcre_exec(p, NULL, s, strlen(s), 0, 0,
                    ovector, sizeof ovector / sizeof *ovector);

    printf(ret < 0 ? "no match\n" : "match\n");
}
$ cc -lpcre pcre.c && ./a.out
match

Susam's Python Pages

Mark V. Shaney Junior 0.2.0

Fed 24 Years of My Blog Posts to a Markov Model

Contents

Recreational Programming

Gibberish

The Markov Property

Some More Gibberish

Mark V. Shaney Junior 0.1.0

Fizz Buzz with Cosines

Contents

Definitions

Symbol Functions

Index Function

Fizz Buzz Sequence

From Indicator Functions to Cosines

Indicator Functions

Complex Exponentials

Cosines

Discrete Fourier Transform

One Period of Fizz Buzz

Fourier Coefficients

Inverse Transform

Conclusion

Elliptical Python Programming

Negative Lookahead Assertion

Import Readline

Toy REPL

Line Editing and History

History File

Readline Wrapper

Obligatory Joke

Unix Timestamp 1600000000

Unix Timestamp Conversion

Python

GNU date (Linux)

BSD date (macOS, FreeBSD, OpenBSD, etc.)

Other Such Dates

Update

Peculiar Self-References

Peculiar Results

The Language Reference

More Experiments

Fixed Bits of Version 4 UUID

Zero Length Regular Expression

GNU grep

BSD grep

Perl

Python

Java

MzScheme

CLISP

C