PLAYPEN Chapter 0: Compiler Setup

Staging

Create a folder for storing the elements you will need throughout the process. My staging folder was simply “C:\Staging”

In the folder created for staging do the following:

1. Open the folder created for staging
2. Right Click -> New -> Shortcut
3. Supply “cmd.exe” as the location of the shortcut
4. Click “Next”
5. Name the shortcut “cmd”
6. Click “Finish”
7. Right Click -> New -> Shortcut
8. Supply “powershell.exe” as the location of the shortcut
9. Click “Next”
10. Name the shortcut “powershell”
11. Click “Finish”

Command Line

While it may not be necessary, my instructions will make use of the command line often. It is easier to give typed instructions than explain the navigation through menus and buttons via text. I will limit this when I can.

Install Chocolatey

Chocolatey is a software package management tool for Windows. It is like Homebrew for Mac or yum, apt, rpm for Linux distributions. To install Chocolatey Windows Powershell must be run as an administrator. So, in your staging folder, right click on the “powershell” shortcut and select “Run as administrator.” From here there are to options: 1) The installation process can be run interactively (“AllSigned”), meaning you accept or deny every option presented. 2) The other option is to force acceptance (“bypass”) for each option without questioning the user.

#Option 1
 Set-ExecutionPolicy AllSigned -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString(‘https://chocolatey.org/install.ps1’))
#Option 2
 Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString(‘https://chocolatey.org/install.ps1’))

Install Tools

There are four key pieces of software need for building the LLVM toolchain. LLVM is what will be used to transform our custom language into assembly. LLVM formerly stood for Low-Level Virtual Machine. I said “formerly” because the website (llvm.org) now says it is not an acronym for anything. From what other websites have said, this is due to how LLVM has expanded and deviated from its initial purpose.

The first software tool is Git. It will be used to obtain LLVM from their source repository. Secondly, CMake, which is a build configuration tool. It manages the build process in a manner that is independent of the compiler. Next, WinFlexBison, which is the lexer and parser. Flex is the lexer used to identify character sequences which then sends meaningful tokens to the parser, Bison. Finally, MinGW-w64 is the compiler toolchain I use to build LLVM.

To install these software packages open a cmd shell window as an administrator and issue the following commands for the respective software package.

#Install git
choco install git.install --params "/GitAndUnixToolsOnPath /NoGitLfs /SChannel /NoAutoCrlf"
#Install cmake
choco install cmake --installargs 'ADD_CMAKE_TO_PATH=User'
#Install Windows flex and bison
choco install winflexbison
#Install Minimal GNU Windows compiler suite
choco install mingw

Optionally, I installed Microsoft Visual Studio Code. I like it because it is cross platform and more importantly (to me) it provides the ability to display the IDE with a command line shell (any shell installed on your system) on the same screen. I assume that isn’t an important feature for most people. Any IDE that supports C/C++ development should be fine, but I will continue my instructions as if the reader is using Microsoft Visual Studio Code for those who need to follow the steps exactly.

Create Project Environment

Download the skeleton code for PlayPen here. Unzip the file in the location of your choice, although I strongly recommend against unzipping it into a path that contains a space. For the sake of the example, I will open the PlayPen.zip file in C:\Staging\. Once unzipped, the extracted folder will have provided the project head with the desired structure

To load PlayPen in Visual Studio Code select “Open Folder” under the “File” menu or use the macro, Ctrl+K, Ctrl+O. Navigate to the PlayPen folder and click the “Select Folder” button. NOTE: You just need to click the folder itself, do not enter it.

The Makefile

Open the file entitled “Makefile.” The first three assignments after the license require that g++.exe, win_flex.exe, win_bison.exe be in your path. If not, you should either put them in your path or include the path with the executable (e.g. C:\ProgramData\chocolatey\bin\g++.exe). Unfortunately, I have no way of knowing where things are in your path. I can tell you that I used Chocolatey when I prepped this instructional, and it seemed to put my executables in C:\ProgramData\chocolatey\bin\.

.toy Files

.toy is the extension I chose for PlayPen. It was a arbitrary decision. The actual extension is meaningless at this point. Below are the contents of the two sample toy files. Right now there isn’t much to explain, but it will evolve. The syntax is as follows: the keyword fun denotes a function followed by the name of the function. Next are the parameters to the function within parenthesis then an arrow pointing to the return type. Following the return type is the block of code. Right now all that is supported is variable declarations.

fun entry: (void) -> void
{
    integer fuzzy
    float wuzzy
    string name
}

fun LittleEngine: (integer ICan) -> float
{
    string message
}

fun BigEngine: (integer ICannot) -> float
{
    integer speed
}

Compiling PlayPen

The PlayPen skeleton source code comes ready to compile as along as the g++.exe, win_flex.exe, win_bison.exe and mingw32-make.exe (in the same location as g++.exe) are in your path. If you installed with Chocolatey, they should be. Otherwise, tweak the Makefile as mentioned above. To build the PlayPen

C:\Users\Syndane> cd \Staging\PlayPen\
C:\Users\Syndane\PlayPen>mingw32-make

Executing PlayPen

To run PlayPen, simply type “bin\PlayPen” and supply a .toy file to compile. Based on the syntax of the input file (.toy), the output of PlayPen should tell you “The input is valid.” or “The input is invalid.”

C:\Staging\PlayPen> bin\PlayPen toybox\teddybear.toy
The input is valid.

Add that’s the first step getting the parser to identify the syntax. I’ll go over this in greater depth, and demonstrate how to add to it or modify it soon

Building LLVM

As mentioned earlier, LLVM is a low level toolchain used to generate the assembly code that will be assembled and linked into an executable. It seems to me that in order to get everything needed to build a custom language/compiler you need to obtain the source and build the libraries. I’ve tried to install LLVM before and didn’t get the development libraries or the header files.

Open cmd shell as an administrator and follow these steps.

Obtain LLVM From Git

C:\Staging\PlayPen\>cd c:\Staging\
C:\Staging>git.exe clone –config core.autocrlf=false https://github.com/llvm/llvm-project.git && cd llvm-project

Downgrade to version 9.0.1

(that’s version I could get working, version 10.0.0 is the highest at the time of this writing (1/28/2020))
C:\Staging\llvm-project>git.exe checkout llvmorg-9.0.1

Create a build directory and enter it

C:\Staging\llvm-project>mkdir build && cd build

Create the Build Configuration With CMake

The CMake options depend on a number of factors (e.g. level of debug info, architectures supported, compiler specification). For information on options follow the link above. C:\Staging\llvm-project\build>b>cmake.exe -G “MinGW Makefiles” ../llvm
[NOTE 1:This is a minimal CMake configuration (all but one default) This works on a Windows Platform using MinGW and my Makefile.]
[NOTE 2: Make sure git is not in your path at this time.]

Build LLVM

C:\Staging\llvm\project\build>b>mingw32-make.exe -j#
# is the number of processors/cores you want to dedicate the “make” process.
[e.g. mingw32-make -j4 (for 4 processors tasked with building LLVM)]
[e.g. mingw32-make (for 1 processor tasked with installing LLVM)]

Install LLVM

C:\Staging\llvm\project\build>mingw32-make.exe install -j#

---

LLVM should install to C:\Program Files\LLVM

Next

We we will examine the flex and bison files. I will explain how the language processing works and how to modify the files to shape your own language.

---

---