Default mapping from ElasticsearchPublisher (TABLE_INDEX_MAP) doesn't work for me #2157
Unanswered
vpirogovrep
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
i try to import my metadata from apache atlas into amundsen via ElasticsearchPublisher like this
job_config = ConfigFactory.from_dict({
'extractor.atlas_search_data.{}'.format(AtlasSearchDataExtractor.ATLAS_URL_CONFIG_KEY): atlas_host,
'extractor.atlas_search_data.{}'.format(AtlasSearchDataExtractor.ATLAS_PORT_CONFIG_KEY): 21000,
'extractor.atlas_search_data.{}'.format(AtlasSearchDataExtractor.ATLAS_VALIDATE_SSL_CONFIG_KEY): True,
'extractor.atlas_search_data.{}'.format(AtlasSearchDataExtractor.ATLAS_USERNAME_CONFIG_KEY): atlas_user,
'extractor.atlas_search_data.{}'.format(AtlasSearchDataExtractor.ATLAS_PASSWORD_CONFIG_KEY): atlas_password,
'extractor.atlas_search_data.{}'.format(AtlasSearchDataExtractor.ENTITY_TYPE_KEY): 'Table',
'loader.filesystem.elasticsearch.{}'.format(
FSElasticsearchJSONLoader.FILE_PATH_CONFIG_KEY): extracted_search_data_path,
'publisher.elasticsearch.{}'.format(ElasticsearchPublisher.FILE_PATH_CONFIG_KEY): extracted_search_data_path,
'publisher.elasticsearch.{}'.format(ElasticsearchPublisher.FILE_MODE_CONFIG_KEY): 'r',
'publisher.elasticsearch.{}'.format(
ElasticsearchPublisher.ELASTICSEARCH_CLIENT_CONFIG_KEY): elasticsearch_client,
'publisher.elasticsearch.{}'.format(
ElasticsearchPublisher.ELASTICSEARCH_NEW_INDEX_CONFIG_KEY): elasticsearch_new_index_key,
'publisher.elasticsearch.{}'.format(
ElasticsearchPublisher.ELASTICSEARCH_DOC_TYPE_CONFIG_KEY): elasticsearch_doc_type_key,
'publisher.elasticsearch.{}'.format(
ElasticsearchPublisher.ELASTICSEARCH_ALIAS_CONFIG_KEY): elasticsearch_index_alias,
'publisher.elasticsearch.{}'.format(
ElasticsearchPublisher.ELASTICSEARCH_MAPPING_CONFIG_KEY): TABLE_INDEX_MAP})
but face an error
elasticsearch.exceptions.RequestError: RequestError(400, 'mapper_parsing_exception', 'Root mapping definition has unsupported parameters: [schema : {analyzer=simple, type=text, fields={raw={type=keyword}}}] [cluster : {analyzer=simple, type=text, fields={raw={type=keyword}}}] [description : {analyzer=simple, type=text}] [display_name : {type=keyword}] [column_descriptions : {analyzer=simple, type=text}] [programmatic_descriptions : {analyzer=simple, type=text}] [tags : {type=keyword}] [badges : {type=keyword}] [database : {analyzer=simple, type=text, fields={raw={type=keyword}}}] [total_usage : {type=long}] [name : {analyzer=simple, type=text, fields={raw={type=keyword}}}] [last_updated_timestamp : {format=epoch_second, type=date}] [unique_usage : {type=long}] [column_names : {analyzer=simple, type=text, fields={raw={normalizer=column_names_normalizer, type=keyword}}}] [key : {type=keyword}]')
my json looks like this
{"badges": [],
"cluster": "",
"column_descriptions": [],
"column_names": ["version", "key", "value"],
"database": "system", "description": null, "display_name":
"default.test_low_cor", "key": "system://.default/test_low_cor",
"last_updated_timestamp": 1685559885, "name": "test_low_cor",
"programmatic_descriptions": [], "schema": "default", "schema_description": "", "tags": [], "total_usage": 0, "unique_usage": 0}
please help
thx
Beta Was this translation helpful? Give feedback.
All reactions