Skip to content

Troubleshooting

No data arriving in the dashboard

Check 1: Is the agent running?

Docker:

docker compose ps
docker compose logs agent | tail -50

systemd:

sudo systemctl status logystera-agent
sudo journalctl -u logystera-agent -n 50

Check 2: Is Vault actually writing audit logs?

ls -lh /var/log/vault/audit.log
tail -f /var/log/vault/audit.log

The file must exist and grow when Vault activity occurs. If it is empty or missing, enable the audit device:

vault audit enable file file_path=/var/log/vault/audit.log

Check 3: Can the agent read the log file?

Docker:

docker compose exec agent ls -la /var/log/vault/

The file must be visible inside the container. If not, check that HOST_VAULT_LOG_DIR in .env points to the correct host directory.

systemd:

sudo -u logystera-agent cat /var/log/vault/audit.log | head -1

If this fails with Permission denied, add the agent user to the vault group:

sudo usermod -aG vault logystera-agent
sudo systemctl restart logystera-agent

Authentication errors (401 / 403)

The agent logs client error 401 or client error 403 from the gateway.

Cause: Invalid or mismatched credentials.

Fix: 1. Go to the Logystera dashboard → Entity → Credentials 2. Verify the Entity Token and Entity Secret match what is in your .env or agent.env 3. Token and secret are case-sensitive — copy them exactly, with no leading/trailing spaces

Gateway unreachable

The agent logs connection errors or timeouts.

Check outbound connectivity:

curl -v https://gateway.logystera.com/v1/ingest

Expected: a 405 Method Not Allowed (the endpoint requires POST). Any other error indicates a network issue.

Common causes: - Firewall blocking outbound port 443 - Restrictive egress rules in the cloud environment (security groups, NACLs) - DNS resolution failure for gateway.logystera.com

The agent retries indefinitely with exponential backoff (up to 5 minutes between retries). Events are buffered on disk at /var/log/logystera-agent/gateway-buffer and will be delivered when connectivity is restored.

Agent starts but no batches in the Ingestion tab

Cause: The agent is running but the log file is not growing.

Vault only writes to the audit log when activity occurs. Generate some activity:

vault token lookup
vault status

Then check the agent logs again — you should see batch sent within 5 seconds.

Position file issues (events being re-sent after restart)

The agent tracks its position in the log file via a .pos file. If the position file is lost or reset, the agent will re-read from the beginning of the current log file.

Docker: the position file is stored in the logystera-agent-data Docker volume. If the volume is deleted, position is lost. Re-sending historical events is harmless — Logystera deduplicates at the processor.

systemd: position file is at /var/log/logystera-agent/audit.pos.

Buffer disk usage growing

If the gateway is unreachable for an extended period, the disk buffer grows. The default limit is 512MB. If this limit is reached, new events are dropped.

Check buffer size:

du -sh /var/log/logystera-agent/gateway-buffer/

The buffer will drain automatically once the gateway is reachable again.

To reduce the limit (lower risk on small disks), edit fluent.conf:

<buffer>
  total_limit_size 64m
</buffer>

Log level: increase verbosity

To see per-batch debug output, set log level to debug in fluent.conf:

<system>
  log_level debug
</system>

This shows each batch being sent, including event counts and response codes. Disable for production — it is verbose.

Check agent version

Docker:

docker compose exec agent cat /opt/logystera-agent/VERSION

systemd:

logystera-agent --version