How to Build A Shell From Scratch (1)

I’ve just hand in the homework of Linux Operating System about creating a basic shell today. The most of functions are done two weeks ago but the requirement about details such as blank ignorance and pipe force me to reconsider the design of code structure again and again.

This tutorial, as a review of work, introduce how to use the fundamental Unix system call to build a useful shell with background, redirection, and pipe supporting. The source code is open at Github.

Why bother building a naive shell when there already are so many great shells like fish, bash and zsh? First, shell is a great window to learn about the fundamental Unix concepts, including process model, process communication, and file operation, to be more specific, and the usage of system call like fork(), exec(), wait(), pipe().

Second, in many cases there is not practical benefit to create a general purpose shell, but a command line user interface is very common, like the interactive command interpreter provided by Python, node.js, and gdb, all of them can be regarded as the domain specific shell.

In summary, learning about how to make a shell is quite useful theoretically and practically, so let’s build it!

2. Main Loop

A shell continue to consume line by line from user until exit. Hence the framework of a shell must be a loop that keep processing inputs from standard input based on line:

#include <stdio.h>

int main() {
    // We assume the input is less than 1024 chars.
    char input[1024];
    char* r;

    while((r = gets(input)) != NULL) {
        printf("%s\n", r);
    }

    return 0;
}

where we use function gets to read from standard input until it reaches the newline \n and store the string in input by removing \n beforehand. Try this by complie and run the executable:

gcc sh.c -o sh
./sh
some input
some input

For now we receive input and print it out as the same. Not quite useful, of course. Next we might try to response to some simple command, like pwd that prints the current working directory and exit to quit, this is simple by invoking the getcwd system call and return:

#include <stdio.h>
#include <string.h>
#include <unistd.h>

int main(){
    char input[1024];
    char* r;

    // use prompt to mark the executing program
    printf("> ");       
    while((r = gets(input)) != NULL) {
        if (strcmp(r, "pwd") == 0) {
            char cwd[80];
            printf("%s\n", getcwd(cwd, sizeof(cwd)));
        }
        if (strcmp(r, "exit") == 0) {
            return 0;
        }
        printf("> ");
    }

    return 0;
}

This time we just simply ignore all other commands. We also added the prompts to mark a shell environment and distinguish the inputs from outputs (You can give your shell a name by placing it before > like resh >!).

3. Parameters Parser

For now we are only able to compare the single command, while in real world there are multiple parameters passed into a command, hence a parser that split the input into parameters is required. To organize the codes, we extract the parsing and command execting process into seperate function, named parse() and route(), and store the parameters in params:

int main(){
    char input[1024];
    char* params[128];
    char* r;

    printf("> ");       
    while((r = gets(input)) != NULL) {
        parse(input, params);
        route(params);
        printf("> ");
    }

    return 0;
}

We use strtok() in parser(), which is designed to split the string into tokens by calling repeatly and returns the start pointer of one token in each call. The first call requires the start pointer of string as the first argument, while NULL for latter calls. Refer to the manual entry of strtok() for more details.

int parse(char* input, char** params){
    int index = 0;

    char* param;

    params[index++] = strtok(input, " ");
    while ((param = (strtok(NULL, " ")))){
        params[index++] = param;
    }

    params[index] = NULL;
    return index;
}

int route(char** params){
    char* cmd = params[0];

    if (strcmp(cmd, "echo") == 0) {
        int index = 1;
        while(params[index]) {
            printf("%s ", params[index]);
            index++;
        }
        printf("\n");
    }

    if (strcmp(cmd, "pwd") == 0) {
        char cwd[80];
        printf("%s\n", getcwd(cwd, sizeof(cwd)));
    }

    if (strcmp(cmd, "exit") == 0) {
        // exit(0) to exit from the process, 
        // instead of just return to main function.
        exit(0);
    }

    return 0;
}

Notice that our echo does not simply print the original string but parse it first, as a result any redundant blanks between parameters will be omitted:

gcc sh.c -o sh
./sh
> echo     1    2    3
1 2 3

For users, it is more common to invoke some powerful programs instead of those shell built-in, such as find, grep. That is, we must empower our shell the ability to execute other programs, which require system calls fork() and exec(). We’ll introduce these in next section, and equip our shell with the ability of all executables.

How to Build A Shell From Scratch (1)

1. Introduction

2. Main Loop

3. Parameters Parser