MCMC and Bayesian statistics in BASIC

BASIC was one of the most common programming languages. In the 80s, it came in a standard set of programs on a computer (for example, Commodore 64 and Apple II), and in the 90s both DOS and Windows 95 included QBasic IDE.

QBasic was also my first programming language. I have not programmed in BASIC for almost 20 years and decided to recall this really strange language. Since I spent a lot of time on Bayesian algorithms , I thought it would be interesting to see how Bayesian analytics would look like in a utility that was 20 years old.



')
Here we implement the Metropolis-Hastings algorithm , the standard Monte Carlo method according to the Markov chain scheme (MCMC), which is used in Bayesian models, in BASIC.

I applied the Laplace distribution to the most fascinating data set: the number of wolves per flock. The sample consisted of 16 packs. As a result, I calculated and displayed the result, still using BASIC ( source ).

Where to get BASIC

There are many different versions of BASIC, but I took the one that grew on: Microsoft QBasic 1.1. QBasic has many advanced functions such as user-defined types and (attention!) Functions. But I didn’t use any additional preferences, because I was going to write old-school BASIC, using lowercase numbering and GOTO, which means that the code can be easily imported into, say, Commodore 64 BASIC.

Download QBasic here . Plus, it's free. Unless of course you are still using DOS, then the next step is to install the DOSBox emulator . After you run QBASIC.EXE, a very friendly, bright blue interface will appear to your look. It can be tested using a custom script. It will clear the screen and type “HELLO WORLD”.



Please note that line numbering is not so important in QBasic, but in older BASIC (for example, because it was on Commodore 64) it is necessary because the program is executed in order.

What we will implement

We will implement the Metropolis-Hastings algorithm on the classic MCMC , which you will encounter first if you study computational methods for Bayesian models.

The Bayesian model that we are going to present has a simple one-dimensional Laplace distribution (just to give the Gaussian distribution a rest). The Laplace distribution, like the Gaussian, is of course symmetrical, but has a sharper peak and gentle tails.

It has two parameters: the shift parameter μ, which also determines its average value and the median and span b, which determines the width of the distribution.



In the Laplace class distribution, the median is the maximum probability estimate for the parameter μ. To include this in the simplest Bayesian model, we must have two parameters. Here I will simplifyly apply Uniform (−∞, ∞) to μ and log (b), this, P (μ), P (log (b)) 1. Full model:

x∼Laplace (μ, b)
μ∼Uniform (−∞, ∞)
log (b) ∼Uniform (−∞, ∞)

I repeat that the data that we will use may be the nicest I have worked with. They contain the number of wolves in a sample of 16 wolf packs (source):

Implementation in R

Before we dive into BASIC, we implement everything in R:

#     x <- c(5, 8, 7, 5, 3, 4, 3, 9, 5, 8, 5, 6, 5, 6, 4, 7) # The log posterior density of the Laplace distribution model, when assuming # uniform/flat priors. The Laplace distribution is not part of base R but is # available in the VGAM package. model <- function(pars) { sum(VGAM::dlaplace(x, pars[1], exp(pars[2]), log = TRUE)) } # The Metropolis-Hastings algorithm using a Uniform(-0.5, 0.5) proposal distribution metrop <- function(n_samples, model, inits) { samples <- matrix(NA, nrow = n_samples, ncol = length(inits)) samples[1,] <- inits for(i in 2:n_samples) { curr_log_dens <- model(samples[i - 1, ]) proposal <- samples[i - 1, ] + runif(length(inits), -0.5, 0.5) proposal_log_dens <- model(proposal) if(runif(1) < exp(proposal_log_dens - curr_log_dens)) { samples[i, ] <- proposal } else { samples[i, ] <- samples[i - 1, ] } } samples } samples <- metrop(n_samples = 1000, model, inits = c(0,0)) # Plotting a traceplot plot(samples[,1], type = "l", ylab = expression(Location ~ mu), col = "blue") # Calculating median posterior and 95% CI discarding the first 250 draws as "burnin". quantile(samples[250:1000,1], c(0.025, 0.5, 0.975)) 

 ## 2.5% 50% 97.5% ## 4.489 5.184 6.144 

(This script simply shows what should happen in BASIC. If you want to try the Metropolis-Hastings method on MCMC in R, use the MCMCmetrop1R in the MCMCpack package , or this Metropolis-Hastings script .

Model in BASIC

Let's start with cleaning the screen ( CLS
CLS
) and variable definitions. DIM
defines arrays and matrices.



Even in this short piece of code, you can find a whole bunch of oddities:

• As we try to stick to the old school, there is line numbering everywhere. It is not added automatically, but prescribed by me, and sometimes I skip the lines. Hence, a good life hack is added to the line by 10 each time so that you can insert the number into the missing line and not rewrite the rest (for example, 75
75
in the screenshot). Since there are no functions in the old BASIC, the line numbering will be used by GOSUB
commands GOSUB
GOSUB
and GOTO.
GOTO.

• All operators are written in upper case, and if you write an operator in lower case, it will be corrected to the upper one.
• Why SAMPLES!
SAMPLES!
In QBasic, all variables are numbers and to change their type you need to add a symbol to the variable name. So THIS$
THIS$
This is a string, and THAT!
THAT!
Is a floating point number. Since the parameters are continuous, most of the variables will end in "!".
• DATA
command DATA
- This is a good way to put data into the program, you can later retrieve it using the READ.
function READ.
READ.
If you have more than one data set, good luck.

We continue the definition of the model. We will present it as a subroutine that ends with the expression RETURN
RETURN
. It is caused by the expression GOSUB
GOSUB
,
, RETURN
, GOSUB.
-, . , . , . , PARAMS!
LOGDENS!.
READ X!,
DATA
RESTORE
READ.




-. , GOSUB 520,
520. RND,
0 1.




:



SAMPLES!
, 95 .



: PRINT "HELLO", "YOU!" HELLO YOU!, PRINT "HELLO"; "YOU!" HELLOYOU! "," ";" .

, , . , . , 320 x 200. : LINE (, )-(, ).




:




, .



GOTO
, . 33 MHz, 386 ( ), , . :





, , 4-5 6. , BASIC R :



( MCMCDEMO.BAS ),


, 20 . GOTO
GOSUB
- . ( END IF
NEXT I.
) 386 . , .

,
, RETURN
, GOSUB.
-, . , . , . , PARAMS!
LOGDENS!.
READ X!,
DATA
RESTORE
READ.




-. , GOSUB 520,
520. RND,
0 1.




:



SAMPLES!
, 95 .



: PRINT "HELLO", "YOU!" HELLO YOU!, PRINT "HELLO"; "YOU!" HELLOYOU! "," ";" .

, , . , . , 320 x 200. : LINE (, )-(, ).




:




, .



GOTO
, . 33 MHz, 386 ( ), , . :





, , 4-5 6. , BASIC R :



( MCMCDEMO.BAS ),


, 20 . GOTO
GOSUB
- . ( END IF
NEXT I.
) 386 . , .

,
, RETURN
, GOSUB.
-, . , . , . , PARAMS!
LOGDENS!.
READ X!,
DATA
RESTORE
READ.




-. , GOSUB 520,
520. RND,
0 1.




:



SAMPLES!
, 95 .



: PRINT "HELLO", "YOU!" HELLO YOU!, PRINT "HELLO"; "YOU!" HELLOYOU! "," ";" .

, , . , . , 320 x 200. : LINE (, )-(, ).




:




, .



GOTO
, . 33 MHz, 386 ( ), , . :





, , 4-5 6. , BASIC R :



( MCMCDEMO.BAS ),


, 20 . GOTO
GOSUB
- . ( END IF
NEXT I.
) 386 . , .

,
, RETURN
, GOSUB.
-, . , . , . , PARAMS!
LOGDENS!.
READ X!,
DATA
RESTORE
READ.




-. , GOSUB 520,
520. RND,
0 1.




:



SAMPLES!
, 95 .



: PRINT "HELLO", "YOU!" HELLO YOU!, PRINT "HELLO"; "YOU!" HELLOYOU! "," ";" .

, , . , . , 320 x 200. : LINE (, )-(, ).




:




, .



GOTO
, . 33 MHz, 386 ( ), , . :





, , 4-5 6. , BASIC R :



( MCMCDEMO.BAS ),


, 20 . GOTO
GOSUB
- . ( END IF
NEXT I.
) 386 . , .

,
, RETURN
, GOSUB.
-, . , . , . , PARAMS!
LOGDENS!.
READ X!,
DATA
RESTORE
READ.




-. , GOSUB 520,
520. RND,
0 1.




:



SAMPLES!
, 95 .



: PRINT "HELLO", "YOU!" HELLO YOU!, PRINT "HELLO"; "YOU!" HELLOYOU! "," ";" .

, , . , . , 320 x 200. : LINE (, )-(, ).




:




, .



GOTO
, . 33 MHz, 386 ( ), , . :





, , 4-5 6. , BASIC R :



( MCMCDEMO.BAS ),


, 20 . GOTO
GOSUB
- . ( END IF
NEXT I.
) 386 . , .

,
, RETURN
, GOSUB.
-, . , . , . , PARAMS!
LOGDENS!.
READ X!,
DATA
RESTORE
READ.




-. , GOSUB 520,
520. RND,
0 1.




:



SAMPLES!
, 95 .



: PRINT "HELLO", "YOU!" HELLO YOU!, PRINT "HELLO"; "YOU!" HELLOYOU! "," ";" .

, , . , . , 320 x 200. : LINE (, )-(, ).




:




, .



GOTO
, . 33 MHz, 386 ( ), , . :





, , 4-5 6. , BASIC R :



( MCMCDEMO.BAS ),


, 20 . GOTO
GOSUB
- . ( END IF
NEXT I.
) 386 . , .

,
, RETURN
, GOSUB.
-, . , . , . , PARAMS!
LOGDENS!.
READ X!,
DATA
RESTORE
READ.




-. , GOSUB 520,
520. RND,
0 1.




:



SAMPLES!
, 95 .



: PRINT "HELLO", "YOU!" HELLO YOU!, PRINT "HELLO"; "YOU!" HELLOYOU! "," ";" .

, , . , . , 320 x 200. : LINE (, )-(, ).




:




, .



GOTO
, . 33 MHz, 386 ( ), , . :





, , 4-5 6. , BASIC R :



( MCMCDEMO.BAS ),


, 20 . GOTO
GOSUB
- . ( END IF
NEXT I.
) 386 . , .

,
, RETURN
, GOSUB.
-, . , . , . , PARAMS!
LOGDENS!.
READ X!,
DATA
RESTORE
READ.




-. , GOSUB 520,
520. RND,
0 1.




:



SAMPLES!
, 95 .



: PRINT "HELLO", "YOU!" HELLO YOU!, PRINT "HELLO"; "YOU!" HELLOYOU! "," ";" .

, , . , . , 320 x 200. : LINE (, )-(, ).




:




, .



GOTO
, . 33 MHz, 386 ( ), , . :





, , 4-5 6. , BASIC R :



( MCMCDEMO.BAS ),


, 20 . GOTO
GOSUB
- . ( END IF
NEXT I.
) 386 . , .

,
, RETURN
, GOSUB.
-, . , . , . , PARAMS!
LOGDENS!.
READ X!,
DATA
RESTORE
READ.




-. , GOSUB 520,
520. RND,
0 1.




:



SAMPLES!
, 95 .



: PRINT "HELLO", "YOU!" HELLO YOU!, PRINT "HELLO"; "YOU!" HELLOYOU! "," ";" .

, , . , . , 320 x 200. : LINE (, )-(, ).




:




, .



GOTO
, . 33 MHz, 386 ( ), , . :





, , 4-5 6. , BASIC R :



( MCMCDEMO.BAS ),


, 20 . GOTO
GOSUB
- . ( END IF
NEXT I.
) 386 . , .

1. ,
, RETURN
, GOSUB.
-, . , . , . , PARAMS!
LOGDENS!.
READ X!,
DATA
RESTORE
READ.




-. , GOSUB 520,
520. RND,
0 1.




:



SAMPLES!
, 95 .



: PRINT "HELLO", "YOU!" HELLO YOU!, PRINT "HELLO"; "YOU!" HELLOYOU! "," ";" .

, , . , . , 320 x 200. : LINE (, )-(, ).




:




, .



GOTO
, . 33 MHz, 386 ( ), , . :





, , 4-5 6. , BASIC R :



( MCMCDEMO.BAS ),


, 20 . GOTO
GOSUB
- . ( END IF
NEXT I.
) 386 . , .
2. ,
, RETURN
, GOSUB.
-, . , . , . , PARAMS!
LOGDENS!.
READ X!,
DATA
RESTORE
READ.




-. , GOSUB 520,
520. RND,
0 1.




:



SAMPLES!
, 95 .



: PRINT "HELLO", "YOU!" HELLO YOU!, PRINT "HELLO"; "YOU!" HELLOYOU! "," ";" .

, , . , . , 320 x 200. : LINE (, )-(, ).




:




, .



GOTO
, . 33 MHz, 386 ( ), , . :





, , 4-5 6. , BASIC R :



( MCMCDEMO.BAS ),


, 20 . GOTO
GOSUB
- . ( END IF
NEXT I.
) 386 . , .
3. ,
, RETURN
, GOSUB.
-, . , . , . , PARAMS!
LOGDENS!.
READ X!,
DATA
RESTORE
READ.




-. , GOSUB 520,
520. RND,
0 1.




:



SAMPLES!
, 95 .



: PRINT "HELLO", "YOU!" HELLO YOU!, PRINT "HELLO"; "YOU!" HELLOYOU! "," ";" .

, , . , . , 320 x 200. : LINE (, )-(, ).




:




, .



GOTO
, . 33 MHz, 386 ( ), , . :





, , 4-5 6. , BASIC R :



( MCMCDEMO.BAS ),


, 20 . GOTO
GOSUB
- . ( END IF
NEXT I.
) 386 . , .
4. ,
, RETURN
, GOSUB.
-, . , . , . , PARAMS!
LOGDENS!.
READ X!,
DATA
RESTORE
READ.




-. , GOSUB 520,
520. RND,
0 1.




:



SAMPLES!
, 95 .



: PRINT "HELLO", "YOU!" HELLO YOU!, PRINT "HELLO"; "YOU!" HELLOYOU! "," ";" .

, , . , . , 320 x 200. : LINE (, )-(, ).




:




, .



GOTO
, . 33 MHz, 386 ( ), , . :





, , 4-5 6. , BASIC R :



( MCMCDEMO.BAS ),


, 20 . GOTO
GOSUB
- . ( END IF
NEXT I.
) 386 . , .
5. ,
, RETURN
, GOSUB.
-, . , . , . , PARAMS!
LOGDENS!.
READ X!,
DATA
RESTORE
READ.




-. , GOSUB 520,
520. RND,
0 1.




:



SAMPLES!
, 95 .



: PRINT "HELLO", "YOU!" HELLO YOU!, PRINT "HELLO"; "YOU!" HELLOYOU! "," ";" .

, , . , . , 320 x 200. : LINE (, )-(, ).




:




, .



GOTO
, . 33 MHz, 386 ( ), , . :





, , 4-5 6. , BASIC R :



( MCMCDEMO.BAS ),


, 20 . GOTO
GOSUB
- . ( END IF
NEXT I.
) 386 . , .