Add fork+exec support, update Indexing.md, and handle "-mrelocation-model pic" flag

[bsdb0y]

Summary
This PR adds support for the --fork_mode flag in mx-index, which provides a fallback mechanism when --reproc_mode fails, particularly useful in Docker environments. This change addresses and resolves issue #577.

Usage
mx-index --target compile_commands.json

Must specify --reproc_mode or --fork_mode.

Indexing can now be run with either of the following modes:

• Using reproc:
  • mx-index --target compile_commands.json --reproc_mode --db sample.db
• Using classic fork+exec as a fallback:
  • mx-index --target compile_commands.json --fork_mode --db sample.db

Additional Fix
While generating compile_commands.json with clang, the flag -mrelocation-model pic also included and cause error. This error occurs because the pic part is mistakenly treated as a filename, resulting in errors like: Unable to find file pic
An internal workaround has been added to handle this, although using gcc to generate the file avoids this issue entirely.

[CLAassistant]

All committers have signed the CLA.

[bsdb0y]

Test1:

//cat sample2.c
#include <stdio.h>

// 1. Function to add two numbers
int add(int a, int b) {
    return a + b;
}

// 2. Function to find the maximum of two numbers
int max(int a, int b) {
    return (a > b) ? a : b;
}

// 3. Function to check if a number is even
int is_even(int n) {
    return (n % 2 == 0);
}

// 4. Function to print a message
void print_hello() {
    printf("Hello, World!\n");
}

// 5. Function to calculate factorial (non-recursive)
int factorial(int n) {
    int result = 1;
    for (int i = 2; i <= n; i++)
        result *= i;
    return result;
}

// Example usage
int main() {
    printf("Add: %d\n", add(5, 3));
    printf("Max: %d\n", max(10, 20));
    printf("Is 4 even? %d\n", is_even(4));
    print_hello();
    printf("Factorial of 5: %d\n", factorial(5));
    return 0;
}

reproc fails inside docker:

(install) root@b5158c58ce21:~/install# mx-index --target compile_commands.json --db sample2.db --reproc_mode
E20250510 02:38:48.142114  6108 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /usr/bin/clang -c -g -O0 -o sample2 sample2.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:38:48.148221  6104 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /usr/lib/llvm-20/bin/clang -cc1 -triple x86_64-pc-linux-gnu -emit-obj -dumpdir sample2- -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debug-info-kind=constructor -dwarf-version=5 -debugger-tuning=gdb -fdebug-compilation-dir=/root/install -fcoverage-compilation-dir=/root/install -resource-dir /usr/lib/llvm-20/lib/clang/20 -internal-isystem /usr/lib/llvm-20/lib/clang/20/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/15/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -O0 -ferror-limit 19 -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -x c -o /tmp/sample2-0daee7.o sample2.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable

(install) root@b5158c58ce21:~/install# ls -alSh sample2.db
-rw-r--r--. 1 root root 3.3M May 10 02:38 sample2.db

fork+exec works

(install) root@b5158c58ce21:~/install# mx-index --target compile_commands.json --db sample2.db --fork_mode

(install) root@b5158c58ce21:~/install# ls -alSh sample2.db
-rw-r--r--. 1 root root 3.4M May 10 02:38 sample2.db

(install) root@b5158c58ce21:~/install# mx-list-functions --db sample2.db
1152921504606847044     2305843009214745037     9281918834641469440     add     def
1152921504606847044     2305843009214745038     9281918834642518016     max     def
1152921504606847044     2305843009214745039     9281918834643566592     is_even def
1152921504606847044     2305843009214745040     9281918834644615168     print_hello     def
1152921504606847044     2305843009214745041     9281918834645663744     factorial       def
1152921504606847044     2305843009214745042     9281918834646712320     main    def

Test2: Real world application

• command: --reproc_mode:
  • mx-index --db /root/OGHarn/demos/zlib/lib.db --target /root/OGHarn/demos/zlib/lib/compile_commands.json --workspace /root/OGHarn/demos/zlib/mx --reproc_mode

Failure logs inside docker:

E20250510 02:30:01.870612  5690 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -D_FILE_OFFSET_BITS=64 -o CMakeFiles/example64.dir/test/example.c.o -c /root/OGHarn/demos/zlib/lib/test/example.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.875167  5691 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -D_FILE_OFFSET_BITS=64 -o CMakeFiles/minigzip64.dir/test/minigzip.c.o -c /root/OGHarn/demos/zlib/lib/test/minigzip.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.883198  5709 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/zutil.c.o -c /root/OGHarn/demos/zlib/lib/zutil.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.891896  5704 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/minigzip.dir/test/minigzip.c.o -c /root/OGHarn/demos/zlib/lib/test/minigzip.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.899298  5706 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/example.dir/test/example.c.o -c /root/OGHarn/demos/zlib/lib/test/example.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.907377  5708 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/zutil.c.o -c /root/OGHarn/demos/zlib/lib/zutil.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.915544  5692 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/uncompr.c.o -c /root/OGHarn/demos/zlib/lib/uncompr.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.924079  5693 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/inflate.c.o -c /root/OGHarn/demos/zlib/lib/inflate.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.931890  5703 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/crc32.c.o -c /root/OGHarn/demos/zlib/lib/crc32.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.939175  5697 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/deflate.c.o -c /root/OGHarn/demos/zlib/lib/deflate.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.953272  5700 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/inftrees.c.o -c /root/OGHarn/demos/zlib/lib/inftrees.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.953497  5705 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/compress.c.o -c /root/OGHarn/demos/zlib/lib/compress.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.969448  5695 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/gzread.c.o -c /root/OGHarn/demos/zlib/lib/gzread.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.969564  5696 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/gzlib.c.o -c /root/OGHarn/demos/zlib/lib/gzlib.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.977866  5702 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/infback.c.o -c /root/OGHarn/demos/zlib/lib/infback.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.987006  5698 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/gzwrite.c.o -c /root/OGHarn/demos/zlib/lib/gzwrite.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:01.996883  5694 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/trees.c.o -c /root/OGHarn/demos/zlib/lib/trees.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.005138  5701 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/gzclose.c.o -c /root/OGHarn/demos/zlib/lib/gzclose.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.013847  5690 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/uncompr.c.o -c /root/OGHarn/demos/zlib/lib/uncompr.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.022713  5699 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/inffast.c.o -c /root/OGHarn/demos/zlib/lib/inffast.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.030362  5697 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/gzclose.c.o -c /root/OGHarn/demos/zlib/lib/gzclose.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.039072  5705 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/crc32.c.o -c /root/OGHarn/demos/zlib/lib/crc32.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.046243  5691 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/trees.c.o -c /root/OGHarn/demos/zlib/lib/trees.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.054956  5709 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/inffast.c.o -c /root/OGHarn/demos/zlib/lib/inffast.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.063756  5692 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/gzwrite.c.o -c /root/OGHarn/demos/zlib/lib/gzwrite.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.071182  5703 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/gzlib.c.o -c /root/OGHarn/demos/zlib/lib/gzlib.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.082054  5707 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -o CMakeFiles/zlibstatic.dir/adler32.c.o -c /root/OGHarn/demos/zlib/lib/adler32.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.090376  5708 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/inflate.c.o -c /root/OGHarn/demos/zlib/lib/inflate.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.100734  5700 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/deflate.c.o -c /root/OGHarn/demos/zlib/lib/deflate.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.109674  5693 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/gzread.c.o -c /root/OGHarn/demos/zlib/lib/gzread.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.118412  5704 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/inftrees.c.o -c /root/OGHarn/demos/zlib/lib/inftrees.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.127548  5695 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/compress.c.o -c /root/OGHarn/demos/zlib/lib/compress.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.138734  5706 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/infback.c.o -c /root/OGHarn/demos/zlib/lib/infback.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable
E20250510 02:30:02.148285  5696 Importer.cpp:693] Error invoking original compiler to find version information: Invalid argument; original command was: /root/AFLplusplus/afl-clang-fast -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -I/root/OGHarn/demos/zlib/lib -fPIC -o CMakeFiles/zlib.dir/adler32.c.o -c /root/OGHarn/demos/zlib/lib/adler32.c. Perhaps try --env /path/to/file to specify a file containing environment variables, to help me find the compiler executable

• command: --fork_mode:
  • mx-index --db /root/OGHarn/demos/zlib/lib.db --target /root/OGHarn/demos/zlib/lib/compile_commands.json --workspace /root/OGHarn/demos/zlib/mx --fork_mode which is success.

[pgoodman]

LGTM!

[pgoodman]

Ping @kumarak. I no longer have reviewer privileges here 😭.

[bsdb0y]

Ping @kumarak. I no longer have reviewer privileges here 😭.

Hi @kumarak, can you please take a look and approve the changes? Let me know if you need anything from me.