Page MenuHomeDevCentral

Collect logs from Docker
Needs ReviewPublic

Authored by dereckson on Sun, Jan 9, 11:30.

Details

Summary

Docker logs -> Filebeat container -> OpenSearch infra-logs

Ref T1624

Test Plan

Collect logs from docker-001

Diff Detail

Repository
rOPS Nasqueron Operations
Lint
Lint Passed
Unit
No Test Coverage
Branch
opensearch
Build Status
Buildable 3834
Build 4084: arc lint + arc unit

Event Timeline

dereckson created this revision.
dereckson added inline comments.
roles/opensearch/opensearch/files/internal_users.yml.jinja
53

We need to create this ingest_client role.

It should be able to:

  • create an index
  • publish documents for any index
roles/paas-docker/containers/filebeat.sls
41
This revision is now accepted and ready to land.Sun, Jan 9, 21:27
dereckson added inline comments.
roles/opensearch/opensearch/files/roles.yml
217

Careful when manually converting JSON to YAML

ERR: Seems /opt/opensearch/plugins/opensearch-security/securityconfig/roles.yml is not in OpenSearch Security 7 format: com.fasterxml.jackson.dataformat.yaml.snakeyaml.error.MarkedYAMLException: while parsing a block collection
 in 'reader', line 217, column 7:
          - "cluster_monitor",
          ^
expected <block end>, but found ','
 in 'reader', line 217, column 26:
          - "cluster_monitor",

roles, remove example config file

This revision is now accepted and ready to land.Sun, Jan 9, 21:39
pillar/paas/docker.sls
149
yaml.composer.ComposerError: found undefined alias
  in "<unicode string>", line 149, column 24

Seems the alias must be defined BEFORE first use.

OpenSearch. OpenSearch part looks good: we've the role and the user.

Filebeat. Docker container runs, load the harvesters correctly, but use http and not https to try to connect to OpenSearch:

{"level":"error","timestamp":"2022-01-09T21:51:06.179Z","logger":"publisher_pipeline_output","caller":"pipeline/output.go:154","message":"Failed to connect to backoff(elasticsearch(http://cloudhugger.nasqueron.org:9200)): Get \"http://cloudhugger.nasqueron.org:9200\": EOF"}

Connect to OpenSearch using https

Connect to OpenSearch using https

{"level":"error","timestamp":"2022-01-09T21:54:40.939Z","logger":"publisher_pipeline_output","caller":"pipeline/output.go:154","message":"Failed to connect to backoff(elasticsearch(https://cloudhugger.nasqueron.org:9200)): Get \"https://cloudhugger.nasqueron.org:9200\": x509: certificate signed by unknown authority"}

Works, but needs multiline configuration. And Logstash as buffer.

Add index policies management