Skip to content

COMPILE.md

Latest commit

 

History

History
27 lines (17 loc) · 637 Bytes

COMPILE.md

File metadata and controls

27 lines (17 loc) · 637 Bytes
Raw

FastTokenize Compilation Guide

Building FastTokenize JAR

Build the standalone Java library using Apache Maven:

mvn clean package -DskipTests

The compiled output will be generated at: target/FastTokenize-0.1.1.jar

Compiling Native C++/AVX2 SIMD Library

To compile the optional native hardware acceleration library (fasttokenize.dll on Windows x64):

  1. Open Visual Studio x64 Native Tools Command Prompt.
  2. Run the build script:
compile.bat

This compiles native/src/fasttokenize_simd.cpp with /arch:AVX2 optimization and outputs src/main/resources/win32-x86-64/fasttokenize.dll.