RANGER-5703: Remove unixauthservice stack and relocate usersync launcher and PAM by ramackri · Pull Request #1105 · apache/ranger

ramackri · 2026-07-23T11:09:30Z

Summary

https://issues.apache.org/jira/browse/RANGER-5703

Remove the remote Unix authentication stack: unixauthservice, unixauthclient, unixauthnative, and unixauthpam (TCP listener on port 5151, RemoteUnixLoginModule, and native validators).
Add UserSyncService as the usersync JVM entry point (HA latch, UserGroupSync, optional metrics) and relocate usersync install scripts/config from unixauthservice/ into ugsync/.
Move local Admin PAM JAAS classes (PamLoginModule, etc.) into security-admin; authentication_method=UNIX and PAM both use local PAM on the Admin host via ranger.pam.service.

Not removed: UserGroupSync (LDAP/file/Unix sources), Admin local PAM (PamLoginModule / libpam4j), LDAP/Kerberos/JDBC auth.

Migration and removal notes: #1105 (comment)

Upgrade impact

Ranger has two separate “Unix” concepts. This PR removes only remote Unix login; it does not remove UserGroupSync or local PAM on Admin.

Concern	What it does	Uses UserSync port 5151?	After this PR
UserGroupSync	Syncs users/groups from LDAP, file, `/etc/passwd`, etc.	No	Unchanged
Admin local PAM	Verifies login on the Admin host via `PamLoginModule` + `ranger.pam.service`	No	Unchanged
Remote Unix login	Admin forwards credentials to UserSync over TCP/TLS	Yes (5151)	Removed

Who is not affected

Deployments using UserGroupSync only (LDAP/file/unix passwd as a sync source)
Deployments using local PAM on Admin (PamLoginModule, /etc/pam.d/... on the Admin node)
Deployments using LDAP, AD, Kerberos, or JDBC Admin authentication

Who is affected (migrate before upgrade)

Sites where Admin login relied on remote Unix auth:

authentication_method=UNIX
ranger.pam.authentication.enabled=false (historical default for UNIX)

That configuration used RemoteUnixLoginModule → UserSync host port 5151 → native validator (credValidator.uexe / pamCredValidator.uexe) on the UserSync node. Password checks ran on UserSync, not on Admin.

Migration options: switch Admin to local PAM (ranger.pam.service on the Admin host) or to LDAP/AD before upgrading.

After this PR, authentication_method=UNIX and PAM both use local PamLoginModule on Admin only.

Test plan

mvn test -pl ugsync -Dtest=org.apache.ranger.usergroupsync.TestUserSyncService
mvn test -pl security-admin -Dtest=org.apache.ranger.security.handler.TestRangerAuthenticationProvider,org.apache.ranger.authentication.unix.jaas.TestPamLoginModule
Build usersync and admin tarballs (mvn package -Pranger-usersync, admin assembly)
Verify usersync starts with UserSyncService main class (not UnixAuthenticationService)
On-prem: confirm Admin PAM login still works with ranger.pam.service configured
Confirm deployments using remote Unix auth (RemoteUnixLoginModule / port 5151) migrate to local PAM or LDAP before upgrade

…her and PAM. Delete unixauthservice, unixauthclient, unixauthnative, and unixauthpam. Add UserSyncService as the usersync JVM entry point, move local PAM JAAS classes into security-admin, and update packaging and install scripts.

ramackri · 2026-07-23T11:26:15Z

Removing the `unixauthservice` Module from Apache Ranger

unixauthservice — UserSync TCP listener
unixauthclient — Ranger Admin JAAS client
unixauthnative / unixauthpam — native validators

Concern	Where it runs	Uses `unixauthservice` TCP (port 5151)?
UserGroupSync (LDAP, file, `/etc/passwd` as a source)	UserSync JVM	No — REST sync to Admin only
Admin local PAM (`PamLoginModule`)	Ranger Admin JVM on the Admin host	No
Remote Unix login (`RemoteUnixLoginModule`)	Admin → UserSync host listener	Yes — removed by this change

Admin local PAM (`PamLoginModule`) — not removed

Uses libpam4j (org.jvnet.libpam.PAM) → pam.authenticate() against /etc/pam.d on the Admin host
Config: ranger.pam.service (PAM service name)
Does not connect to usersync port 5151
Does not run credValidator.uexe / pamCredValidator.uexe
Does not read ranger.usersync.passwordvalidator.path

After removal, PamLoginModule (and related JAAS classes) live under security-admin/, not in deleted unixauthclient/.

authentication_method=UNIX and authentication_method=PAM both use this local path after removal (getPamAuthentication()).

Remote Unix login (`RemoteUnixLoginModule`) — removed

When authentication_method=UNIX and ranger.pam.authentication.enabled=false (historical default for UNIX), Admin used the TCP client:

RemoteUnixLoginModule (in unixauthclient) opens TLS to authServiceHostName:5151
Sends LOGIN: user password to UnixAuthenticationService on the usersync host
PasswordValidator spawns the native binary from ranger.usersync.passwordvalidator.path

That validator runs on the usersync node, not on Admin:

Binary (typical path)	What it does
`credValidator.uexe` (default)	`/etc/shadow` + `crypt()` — OS password check, not PAM
`pamCredValidator.uexe` (if configured)	PAM via `/etc/pam.d/...` on the usersync host

So ranger.usersync.passwordvalidator.path controls the remote listener only. It is not used by Admin PamLoginModule. Removing unixauthservice removes this remote path and those native validators — not local PAM on Admin.

What we delete vs what we keep

Delete	Keep
`unixauthservice` TCP listener	UserGroupSync
`RemoteUnixLoginModule` + `getUnixAuthentication()`	`PamLoginModule` (moved to `security-admin`)
`credValidator.uexe` / `pamCredValidator.uexe` on usersync	Admin LDAP / Kerberos / local PAM / JDBC
`ranger.unixauth.*` (Admin client to listener)	`ranger.pam.service` (local PAM on Admin)

Critical: `UnixAuthenticationService` is also the process launcher

Today UnixAuthenticationService.main() does more than start the TCP auth listener. It is the usersync JVM entry point that:

Initializes UserSyncHAInitializerImpl (Curator leader latch for HA UserSync).
Starts UserGroupSync in a non-daemon background thread.
Optionally starts UserSyncMetricsProducer when ranger.usersync.metrics.enabled=true.
Optionally starts the TCP auth listener when -enableUnixAuth is on the command line (removed entirely — no config toggle replaces it).

Modules to delete

Module	Path	What it does
`unixauthservice`	`unixauthservice/`	TCP listener on port 5151; remote OS password check for Admin login (via native validator on usersync host)
`unixauthclient`	`unixauthclient/`	JAAS client in Ranger Admin — connects Admin UI login to that listener
`unixauthnative`	`unixauthnative/`	Builds `credValidator.uexe` (native password check for listener)
`unixauthpam`	`unixauthpam/`	Builds `pamCredValidator.uexe` (PAM validator for listener)

What `unixauthclient` is (and why it goes too)

Remote Unix login was a two-part feature:

flowchart LR
  subgraph admin [Ranger Admin — remove client]
    A[RangerAuthenticationProvider.getUnixAuthentication]
    B[RemoteUnixLoginModule / unixauthclient]
  end
  subgraph usersync [UserSync JVM — remove listener]
    C[UnixAuthenticationService :5151]
    D[UserGroupSync]
  end
  A --> B
  B -->|TCP| C
  D -->|REST sync| admin

Part	Location	Action
Listener (server)	`unixauthservice` on UserSync host	Delete module
Client	`unixauthclient` in Admin WAR	Delete module — useless without listener
Native validators	`unixauthnative`, `unixauthpam`	Delete modules — only used by listener

Do not remove the listener but keep unixauthclient or ranger.unixauth.* config — that is dead code, not removal.

Do not add ranger.usersync.unix.auth.enabled or any disable toggle — delete the feature.

What stays (unchanged)

Component	Module
LDAP / file / Unix user-group sync	`ugsync/` — `UserGroupSync`, builders, REST to Admin
HA UserSync	`UserSyncHAInitializerImpl` — move init into new `ugsync` launcher
JVM metrics	`UserSyncMetricsProducer` — move start into new launcher
Admin LDAP / Kerberos / local PAM / local DB	`security-admin/` — `PamLoginModule` relocated here; unrelated to deleted TCP client

Changes in `ugsync/`

Required for removal — not optional cleanup.

1. `UserGroupSyncConfig.java`

File: ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java

Change	Detail
Remove auth-listener property constants/accessors if any exist	Port, SSL, password validator path were read by `UnixAuthenticationService`, not by core sync
Keep all sync-related config	LDAP, file, unix min uid/gid, policy manager URL, HA, metrics, deletes, validation, etc.

Tests: TestUserGroupSyncConfig.java — no new toggle tests; verify sync config still loads after relocated ranger-ugsync-default.xml.

2. New usersync JVM entry point (required)

Do not point startup scripts at bare UserGroupSync.main() alone — it does not initialize HA or metrics.

Add a launcher in ugsync/ (e.g. UserSyncService) that replicates the non-auth portion of UnixAuthenticationService:

Responsibility	Source today (`UnixAuthenticationService`)	New launcher
HA init	`UserSyncHAInitializerImpl.getInstance(...)`	Same
Start sync thread	`startUnixUserGroupSyncProcess()` → `UserGroupSync`	Same
Start metrics thread	`UserSyncMetricsProducer` when enabled	Same
Block main thread	Auth `accept()` loop today	Join/wait on sync (or infinite wait) — must not exit and tear down HA in `finally`
TCP auth listener	`init()` + `startService()`	Delete — do not relocate

Tests to add/move: HA + metrics startup tests from unixauthservice/src/test/.../TestUnixAuthenticationService.java → ugsync tests.

3. Other `ugsync/` file edits

File	Change
`UserGroupSync.java`	Optional: fold launcher into `main()` or separate `UserSyncService.java`
`FileSourceUserGroupBuilder.java`	Error message: `unixauthservice.properties` → `ranger-ugsync-site.xml`
`ranger-ugsync-site.xml` (test resources)	Remove `unixauthservice.jks` / auth-only paths
`UserSyncHAInitializerImpl.java`	No API change; launcher must call `getInstance()` before sync
`UserSyncMetricsProducer.java`	No change; launcher starts it

4. Usersync config XML — delete auth properties

File: ranger-ugsync-default.xml (relocated from unixauthservice/conf.dist/)

Remove (auth listener only — no replacement toggle):

Property	Reason
`ranger.usersync.port`	TCP listener port 5151
`ranger.usersync.ssl`	Listener TLS
`ranger.usersync.passwordvalidator.path`	Native validator for listener
`ranger.usersync.unixauth.*`	Rate limits (RANGER-5690) — listener deleted
`ranger.usersync.unix.auth.enabled`	Do not add — not part of removal

Keep:

Property	Reason
`ranger.usersync.enabled`	Core sync on/off
LDAP / file / unix source props	Sync sources
`ranger.usersync.metrics.enabled`	JVM metrics
`ranger-ugsync.server.ha.enabled`	HA
Policy manager URL, sleep intervals, etc.	Core sync

Also update: scripts/templates/ranger-ugsync-template.xml, installprop2xml.properties — remove AUTH_* mappings (AUTH_SSL_*, port, validator); install.properties — remove AUTH_* keys and unixauthservice.jks defaults.

Step 1 — Relocate Assets (Do Not Delete Blindly)

Several paths under unixauthservice/ are not auth-server-only; they are consumed by the usersync distribution. Move them before deleting the module (suggested target: ugsync/ or a new usersync-packaging/ directory).

Relocate from `unixauthservice/conf.dist/`

File	Purpose
`ranger-ugsync-default.xml`	Default usersync config — strip auth listener properties after relocate
`logback.xml`	Usersync logging defaults
`jaas.conf`	Was under `conf.dist/` — delete if only used by removed unixauth stack; verify no other consumer

Relocate from `unixauthservice/scripts/`

File / directory	Purpose
`start.sh`, `stop.sh`	Usersync service control
`setup.sh`, `setup.py`	Usersync install
`install.properties`	Install-time properties template
`ranger-usersync-services.sh`	Change main class to new `ugsync` launcher; remove `-enableUnixAuth` and `UnixAuthenticationService`
`initd`	Init script source for assembly
`templates/`	Config templates
`updatepolicymgrpassword.sh`, `updatepolicymgrpassword.py`, `update_property.py`, `set_globals.sh`	Install/upgrade helpers

Optional / generated

Path	Notes
`unixauthservice/cert/`	Referenced by `distro/src/main/assembly/usersync.xml`; may be empty in source (certs generated at install)

After relocation, update distro/src/main/assembly/usersync.xml so all ../unixauthservice/... paths point to the new location.

Step 2 — Delete Module Directories

Remove entire directories after assets are relocated:

unixauthservice/     # listener + usersync packaging (relocate non-auth assets first)
unixauthclient/      # Admin JAAS client
unixauthnative/      # credValidator native build
unixauthpam/         # pamCredValidator native build

unixauthservice/ contents before delete:
├── pom.xml
├── conf.dist/ (after relocation)
├── scripts/ (after relocation)
├── src/main/java/org/apache/ranger/authentication/
│ ├── UnixAuthenticationService.java
│ ├── PasswordValidator.java
│ └── LoginAttemptTracker.java
├── src/test/java/org/apache/ranger/authentication/
│ ├── TestUnixAuthenticationService.java
│ ├── TestPasswordValidator.java
│ └── LoginAttemptTrackerTest.java
└── src/main/resources/logback.xml


---

The JVM main class in `ps` is always `UnixAuthenticationService`, even though the same process also starts UserGroupSync:

```java
public void run() {
    try {
        LOG.info("Starting User Sync Service!");
        startUnixUserGroupSyncProcess();
        Thread.sleep(5000);
        if (enableUnixAuth) {
            LOG.info("Enabling Unix Auth Service!");
            init();
            startService();
        } else {
            LOG.info("Unix Auth Service Disabled!");
        }

With -enableUnixAuth, the main thread blocks in the auth listener loop, so the process stays up and the grep finds it. Docker CI uses this line only to keep the container alive (tail --pid=...).

What is wrong with it (conceptually)

Issue	Detail
Misleading	Variable is `RANGER_USERSYNC_PID` but the grep matches `UnixAuthenticationService`, not `UserGroupSync`
Coupled to UnixAuth	Usersync lifecycle is tied to a class you may remove
After removal	If startup becomes `UserGroupSync` (or another main), this grep returns empty → container exits → CI fails
Fragile matching	String grep on full command line; renames or package moves break it
Multiple PIDs	If several lines match, `awk '{print $2}'` can yield multiple PIDs and break `tail --pid=`

Better options (especially after removal)

1. Use the PID file (already written by ranger-usersync-services.sh):

pidf=${USERSYNC_PID_DIR_PATH:-/var/run/ranger}/${USERSYNC_PID_NAME:-usersync.pid}
RANGER_USERSYNC_PID=$(cat "$pidf" 2>/dev/null)

2. Grep the usersync process marker (already on the java command line):

RANGER_USERSYNC_PID=$(ps -ef | grep -v grep | grep 'proc_rangerusersync' | awk '{print $2}')

3. After unixauthservice removal, grep the new main class if it becomes the direct entry point, e.g.:

grep -i "org.apache.ranger.usergroupsync.UserGroupSync"

Step 5 — Clean Up References (Optional but Recommended)

`ugsync/` and tests

File	Reference
`ugsync/.../FileSourceUserGroupBuilder.java`	Error message mentions `unixauthservice.properties`
`ugsync/src/test/resources/ranger-ugsync-site.xml`	`unixauthservice.jks` keystore path
`ugsync/src/test/java/.../TestUserGroupSyncConfig.java`	Verify sync config after auth props removed from defaults

Packaging / install (relocated scripts)

File	Reference
`setup.py`	Remove `unixauthservice.jks` cert generation — or delete auth cert setup entirely
`updatepolicymgrpassword.sh`	Rename `unixauthservice.properties` → usersync naming
`install.properties`	Remove `AUTH_*` keys and `unixauthservice.jks` paths
`installprop2xml.properties`	Remove `AUTH_SSL_*` mappings

Docker / dev-support

File	Reference
`dev-support/ranger-docker/scripts/usersync/ranger-usersync-install.properties`	`AUTH_SSL_KEYSTORE_FILE=.../unixauthservice.jks`
`dev-support/ranger-docker/Dockerfile.ranger-usersync`	Symlinks `ranger-usersync-services.sh`; no change unless main class changes
`dev-support/ranger-docker/docker-compose.ranger-usersync.yml`	Mounts install.properties

Migration utilities

File	Reference
`migration-util/ambari2.0-hdp2.2-ranger0.40/bin/import_ranger_to_ambari.py`	`unixauthservice.properties` path
`migration-util/ambari2.1-hdp2.3-ranger0.50/bin/import_ranger_to_ambari.py`	Remove `ranger.usersync.port`, `.ssl`, `.passwordvalidator.path`, `ranger.unixauth.*` mappings

Docs / security

File	Reference
`THREAT_MODEL.md`	Lists `unixauthservice/` under Authentication
`mkdocs/docs/project/cve-list.md`	CVE entries for `UnixAuthenticationService`
`mkdocs/docs/project/release-process.md`	May reference `unixauthnative/pom.xml`
`security-admin/scripts/upgrade_admin.py`	Remove `unixauth.properties` / JAAS migration paths
`unixauthclient/src/test/.../TestRemoteUnixLoginModule.java`	Delete with module
`unixauthclient/src/test/.../UnixAuthenticationTester.java`	Delete with module

Native validators

Module	Action
`unixauthnative/`	Delete — `credValidator.uexe` only served deleted listener
`unixauthpam/`	Delete — `pamCredValidator.uexe` same

Post-Removal Behavior

Component	After removal
UserGroupSync	Continues via new `ugsync` launcher (LDAP, file, Unix passwd/group sources)
UserSync HA / metrics	Works if new launcher initializes HA and starts metrics
Remote Unix Admin login	Removed — `RemoteUnixLoginModule` and listener deleted
Admin `authentication_method=UNIX` or `PAM`	Still supported — local PAM on Admin via `PamLoginModule` in `security-admin` (libpam4j, `ranger.pam.service`)
Admin auth (other)	LDAP, Kerberos, PAM, local DB (`NONE`)
Docker CI usersync	Still works (file-based sync); update PID detection

Verification Checklist

# Reactor should not list unix auth modules
grep -E 'unixauth(service|client|native|pam)' pom.xml distro/pom.xml

# Full build
mvn -pl distro -am package -DskipTests

# Inspect tarballs
tar tzf distro/target/ranger-*-usersync.tar.gz | head
tar tzf distro/target/ranger-*-admin.tar.gz | grep -i unixauth || echo "OK: no unixauth in admin tarball"

# Confirm no UnixAuthenticationService in startup scripts
grep -r UnixAuthenticationService dev-support/ security-admin/src/bin/ <relocated-scripts>/

# Confirm no unixauthclient in Admin
grep -r unixauthclient security-admin/ distro/

Expected:

mvn reactor builds without unixauthservice, unixauthclient, unixauthnative, unixauthpam
Usersync tarball: relocated scripts/config; dist/unixusersync-*.jar + lib/*; no native validators; no auth listener props
Admin tarball: no unixauthclient; no ranger.unixauth.* in defaults
getUnixAuthentication() removed from RangerAuthenticationProvider
New ugsync launcher starts HA + metrics + UserSyncService
Docker usersync container stays up (proc_rangerusersync / UserSyncService)
Docker PAM admin smoke (ranger-admin-pam on :6081, authentication_method=PAM, OS user login)
Release notes: remote Unix login removed, not disabled

The CI services-docker-build job greps for the usersync JVM to keep the container alive; update it from UnixAuthenticationService to proc_rangerusersync after the launcher change.

ramackri force-pushed the remove-unixauth-stack branch from b6ecf36 to d894fd7 Compare July 23, 2026 11:16

ramackri force-pushed the remove-unixauth-stack branch from d894fd7 to 00db543 Compare July 23, 2026 11:24

RANGER-5703: Fix usersync Docker script PID check for UserSyncService.

ec372f9

The CI services-docker-build job greps for the usersync JVM to keep the container alive; update it from UnixAuthenticationService to proc_rangerusersync after the launcher change.

ramackri requested review from dineshkumar-yadav, kumaab, mneethiraj, pradeepagrawal8184, rameeshm, spolavarpau1 and vperiasamy July 23, 2026 15:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

RANGER-5703: Remove unixauthservice stack and relocate usersync launcher and PAM#1105

RANGER-5703: Remove unixauthservice stack and relocate usersync launcher and PAM#1105
ramackri wants to merge 2 commits into
apache:masterfrom
ramackri:remove-unixauth-stack

ramackri commented Jul 23, 2026 •

edited

Loading

Uh oh!

ramackri commented Jul 23, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

ramackri commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Upgrade impact

Who is not affected

Who is affected (migrate before upgrade)

Test plan

Uh oh!

ramackri commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Removing the unixauthservice Module from Apache Ranger

Admin local PAM (PamLoginModule) — not removed

Remote Unix login (RemoteUnixLoginModule) — removed

What we delete vs what we keep

Critical: UnixAuthenticationService is also the process launcher

Modules to delete

What unixauthclient is (and why it goes too)

What stays (unchanged)

Changes in ugsync/

1. UserGroupSyncConfig.java

2. New usersync JVM entry point (required)

3. Other ugsync/ file edits

4. Usersync config XML — delete auth properties

Step 1 — Relocate Assets (Do Not Delete Blindly)

Relocate from unixauthservice/conf.dist/

Relocate from unixauthservice/scripts/

Optional / generated

Step 2 — Delete Module Directories

What is wrong with it (conceptually)

Better options (especially after removal)

Step 5 — Clean Up References (Optional but Recommended)

ugsync/ and tests

Packaging / install (relocated scripts)

Docker / dev-support

Migration utilities

Docs / security

Native validators

Post-Removal Behavior

Verification Checklist

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ramackri commented Jul 23, 2026 •

edited

Loading

ramackri commented Jul 23, 2026 •

edited

Loading

Removing the `unixauthservice` Module from Apache Ranger

Admin local PAM (`PamLoginModule`) — not removed

Remote Unix login (`RemoteUnixLoginModule`) — removed

Critical: `UnixAuthenticationService` is also the process launcher

What `unixauthclient` is (and why it goes too)

Changes in `ugsync/`

1. `UserGroupSyncConfig.java`

3. Other `ugsync/` file edits

Relocate from `unixauthservice/conf.dist/`

Relocate from `unixauthservice/scripts/`

`ugsync/` and tests