hal: Break the HAL API - Move to 64-bit exclusively

[BsAtHome]

This is the long awaited HAL API break. The main update is that all 32-bit HAL pins/params/signals are gone and everything is now 64-bit with backing storage access completely opaque with the previous getter/setter update.

This PR does not include the docs. The PR is already large enough without having to go filter the docs while reviewing. The docs will follow in the next PR and is also quite a large patch set.

If you wonder about the HAL type name change... It was done to make a clean break. It is a clear sign that "from this point on we have a new HAL interface".

What changed:

• HAL library:
  • HAL types HAL_{S32,U32} have been removed. All places where they were used have been updated to use the new abstract HAL_{SINT,UINT}.
  • Old HAL types HAL_{BIT,FLOAT,S64,U64} have been renamed to HAL_{BOOL,REAL,SINT,UINT}. Using any old types will result in a deprecation compiler warning.
  • Old access types like hal_data_u, hal_{bit,float,s32,u32,s64,u64}_t, real_t and ireal_t have been removed. There is no longer direct access to the underlying data. The new HAL types are hal_{bool,real,sint,uint,port}_t require you to use the getter/setter interface.
  • Pin and param direction types hal_pin_dir_t and hal_param_dir_t have been unified into hal_pdir_t. The code has been updated and using the old types will result in a deprecation warning. They must be removed at a (slightly) later stage.
  • The HAL_PORT interface has been changed to use the same interface as the other new HAL types to make the API as consistent as possible.
  • Old pin/param creation functions hal_pin_XXX_new() and hal_param_XXX_new() have been removed. These were already replaced by hal_pin_new_YYYY() and hal_param_new_YYYY()
  • Old parameter set functions hal_param_set*() and the value retrieval functions hal_get_{pin,param.signal}_value_by_name() have been removed. The query API has already replaced these in the previous updates to HAL.
  • The "use floating point" use_fp flag in functions and threads has been removed. It was already deprecated and fixed to true. The RTAPI has been updated as well.
  • The hal.h header now has a version define HAL_API_VERSION and is set to 1. It is now possible to track further changes in the API by inspecting this define to make user's code adapt more easily to future changes.
  • The tests have been updated to the new names so they all pass. It must be noted that tests for 32-bit value wrapping behaviour no longer exhibit that behaviour (specifically tests/halmodule/comp-set-get and tests/mux). If it is required to test wrapping, then the tests may be altered at a later stage.
• HAL Components:
  • abs_s32 and abs_s64 have been folded into abs_sint
  • Old conv_XXXX_YYYY have all been removed and replaced by the new name versions {bool,real,sint,uint}.
  • demux_generic and mux_generic have been updated to handle any-to-any conversion for the new types. The pins that included the old names were also renamed.
  • scaled_s32_sums got renamed to scaled_sint_sums.
  • Component using letters to map types, like streams, will all map to the new HAL types. The old 64-bit identifiers 'l' and 'k' will automatically map to 's' and 'u' respectively for compatibility.
  • A large set of components already has their si32 and ui32 pins/params upgraded to sint and uint. However, there are still many components that need to be made 64-bit clean. This is a larger effort that will follow after integration of the PR. The [su]i32 types do create the correct [su]int typed pins and access through the getter/setter interface. The type marker is present as a reminder that they need updating. However, you cannot change the hal_[gs]et_[su]i32() getters/setters without properly inspecting the code where it is used. Every use needs to be inspected and traced to prevent intermediate variable truncation problems.
  • There are still instances where the s32 or u32 name is used in variables or pin/param/signal names. Not all can be changed easily while others may be replaced. Renaming must be a process that can be done after integrating these changes.
• Utilities and consig samples:
  • halcmd and halrmt now issue a warning when the old HAL types are used. Printing/output data all uses the new types.
  • halshow will properly show the new types
  • All sample configs, where appropriate, were updated to use the new interfaces. Not all sample configs can be tested because some require actual hardware attached.
• Python interfaces:
  • halmodule - The HAL types and pin/param direction have now IntEnums hal.Type.XXXX and hal.Dir.YYY. All code has been updated to use the new types.
  • halmodule - The use of the old HAL_XXX names will result in a deprecation warning. The old 32-bit versions will map to the new types.
  • GUI support library pyvcp now uses tags sint, uint, image_bool and image_uint. The old tags {s32,u32,image_u32,image_bit} are still recognized for compatibility. All in-tree xml files have been updated. The old names are still recognized but will emit a deprecation warning (only once for each type).
  • The pyui panelui now recognize {BOOL,REAL,SINT,UINT} as output type names. All in-tree examples have been updated. The old names are still recognized but will emit a deprecation warning (only once for each type).
• and probably some details I missed in this already very long list...

The changes have been successfully tested on a RPi5+Mesa-7I96S+Modbus.

If you can, you should pull this branch (https://github.com/BsAtHome/linuxcnc/tree/hal64_break-hal-api-nodocs) from my repo and test on your machine/hardware and report back. Not that problems are expected, but testing this is necessary.

[BsAtHome]

Forgot to note: Any hm2_modbus communication mbccb files need to be recompiled from the mbccs source with mesambccc. The file version was increased because of the HAL type changes.

[grandixximo]

CI is green and I went through the diff by area; the things I would have expected to break silently (use_fp callers, pdir sets, the by-value port interface, removed symbols, save/show round trip, the halui pin set, motion locals, MesaMB02, python UINT above 2^63, the GUI compat warnings) all check out.

Three things in halmodule.cc are pre-existing and not from this PR, so I will send a separate PR unless you prefer to fold them in: from_python(PyObject*, bool*) line 150 does *b = 0.0 == v, so a float 1.0 into a bool pin writes False; halenum_rt_members[] at 2167 has no {} terminator and halenum_build("RTType", ...) reads past it; the PyErr_Format at 893 has three conversions and two arguments.

[BsAtHome]

CI is green and I went through the diff by area; the things I would have expected to break silently (use_fp callers, pdir sets, the by-value port interface, removed symbols, save/show round trip, the halui pin set, motion locals, MesaMB02, python UINT above 2^63, the GUI compat warnings) all check out.

Yes, I was amazed myself how few problems I encountered. All the python code is rather transparent to the change. The Tcl code may still present problems because the default internal representation in 8.6 is still 32-bit. It changed to 64-bit in Tcl 9. Luckily, most values are rather small.

There are still a lot of hal_[gs]et_[us]i32() in there. So the code is not 64-bit clean. That part will be a challenge.

Three things in halmodule.cc are pre-existing and not from this PR, so I will send a separate PR unless you prefer to fold them in: from_python(PyObject*, bool*) line 150 does *b = 0.0 == v, so a float 1.0 into a bool pin writes False;

Oops...

halenum_rt_members[] at 2167 has no {} terminator and halenum_build("RTType", ...) reads past it;

This is serious and may give spurious fails on memory layout changes on different compiles.

the PyErr_Format at 893 has three conversions and two arguments.

I thought that there was a printf attribute on the function. Obviously not.

But, yes, these can be fixed in a separate PR (asap). There are no conflicts in that code

[grandixximo]

lib/hallib/hal_procs_lib.tcl:147

get_netlist switches on the type column of hal show signal and only knows float bit u32 s32; anything else hits return -code error "get_netlist: Unexpected <$line>". halcmd now prints bool real sint uint, so hookup_moveoff.tcl (every configs/sim/axis/moveoff/*.ini) dies at load. Add the four new names, or turn the switch around and let anything that is not ==>, <==, <=> set sname?

tcl/bin/halconfig.tcl:752

Same class as the halshow fix you already made ($type == "bit" || $type == "bool"): here and at 986 halconfig still tests for "bit" only, so bool params and pins fall into the text branch instead of the LED. Worth the same two-word patch, or is halconfig on its way out?

Second pass on a9982e2: all eleven points from the first round are in. I merged the head onto current master locally (that brings in #4568 and #4517, both touching files this PR touches): clean merge, warning-free build, runtests 318 run, 317 pass, 3 skipped, the one failure (ui-smoke/qtdragon-quit) is the load flake and passes rerun alone. The get_netlist item is the only thing I found that breaks a shipped config (verified: master OK, this head fails on both signals); the rest of the pass (hal.h/hal_lib.c/hal_priv.h, the conv template bounds, hm2_modbus and mesambccc, halui/motion/inihal NATIVE mapping, the component locals that went 64-bit, stepgen timers, python and qtvcp shims, halcmd/halrmt/halsh/halscope) reads consistent.

[BsAtHome]

lib/hallib/hal_procs_lib.tcl:147
get_netlist switches on the type column of hal show signal and only knows float bit u32 s32;

Added the new types alongside.

tcl/bin/halconfig.tcl:752
Same class as the halshow fix you already made ($type == "bit" || $type == "bool"):

Also fixed.

This Tcl code will eventually be replaced and retired (I hope). Then it should no longer matter ;-)