文库 / 技术

将我的弹性搜索存储库连接到 INCEpTION NER 注释程序的正确 是什么
分享于2022年07月17日 elasticsearch json named-entity-recognition 问答
【问题标题】：将我的弹性搜索存储库连接到 INCEpTION NER 注释程序的正确 是什么(What is the correct to use to connect my elasticsearch repository to the INCEpTION NER annotation program)
【发布时间】：2022-01-22 07:06:36
【问题描述】：

我已经为文本注释设置了本地环境，并希望使用此处开发的 INCEpTION 应用程序： https://github.com/inception-project/inception/blob/main/CONTRIBUTORS.txt

当尝试连接到我的存储库时，我可以使用此处找到的示例连接并查找文档： https://inception-project.github.io/releases/22.1/docs/user-guide.html#sect_external-search-repos

但是，当我尝试连接到使用 FSCrawler 创建和索引的存储库时，我无法进行搜索。

他们的例子的映射是： 

{
  "mappings": {
    "_doc": {
      "properties": {
        "doc": {
          "properties": {
            "text": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "metadata": {
          "properties": {
            "language": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "source": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "timestamp": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "title": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "uri": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        }
      }
    }
  }
}

我的索引映射是： 

{
  "mappings": {
    "_doc": {
      "dynamic_templates": [
        {
          "raw_as_text": {
            "path_match": "meta.raw.*",
            "mapping": {
              "fields": {
                "keyword": {
                  "ignore_above": 256,
                  "type": "keyword"
                }
              },
              "type": "text"
            }
          }
        }
      ],
      "properties": {
        "attachment": {
          "type": "binary"
        },
        "attributes": {
          "properties": {
            "group": {
              "type": "keyword"
            },
            "owner": {
              "type": "keyword"
            }
          }
        },
        "content": {
          "type": "text"
        },
        "file": {
          "properties": {
            "checksum": {
              "type": "keyword"
            },
            "content_type": {
              "type": "keyword"
            },
            "created": {
              "type": "date",
              "format": "dateOptionalTime"
            },
            "extension": {
              "type": "keyword"
            },
            "filename": {
              "type": "keyword",
              "store": true
            },
            "filesize": {
              "type": "long"
            },
            "indexed_chars": {
              "type": "long"
            },
            "indexing_date": {
              "type": "date",
              "format": "dateOptionalTime"
            },
            "last_accessed": {
              "type": "date",
              "format": "dateOptionalTime"
            },
            "last_modified": {
              "type": "date",
              "format": "dateOptionalTime"
            },
            "url": {
              "type": "keyword",
              "index": false
            }
          }
        },
        "meta": {
          "properties": {
            "altitude": {
              "type": "text"
            },
            "author": {
              "type": "text"
            },
            "comments": {
              "type": "text"
            },
            "contributor": {
              "type": "text"
            },
            "coverage": {
              "type": "text"
            },
            "created": {
              "type": "date",
              "format": "dateOptionalTime"
            },
            "creator_tool": {
              "type": "keyword"
            },
            "date": {
              "type": "date",
              "format": "dateOptionalTime"
            },
            "description": {
              "type": "text"
            },
            "format": {
              "type": "text"
            },
            "identifier": {
              "type": "text"
            },
            "keywords": {
              "type": "text"
            },
            "language": {
              "type": "keyword"
            },
            "latitude": {
              "type": "text"
            },
            "longitude": {
              "type": "text"
            },
            "metadata_date": {
              "type": "date",
              "format": "dateOptionalTime"
            },
            "modifier": {
              "type": "text"
            },
            "print_date": {
              "type": "date",
              "format": "dateOptionalTime"
            },
            "publisher": {
              "type": "text"
            },
            "rating": {
              "type": "byte"
            },
            "relation": {
              "type": "text"
            },
            "rights": {
              "type": "text"
            },
            "source": {
              "type": "text"
            },
            "title": {
              "type": "text"
            },
            "type": {
              "type": "text"
            }
          }
        },
        "path": {
          "properties": {
            "real": {
              "type": "keyword",
              "fields": {
                "fulltext": {
                  "type": "text"
                },
                "tree": {
                  "type": "text",
                  "analyzer": "fscrawler_path",
                  "fielddata": true
                }
              }
            },
            "root": {
              "type": "keyword"
            },
            "virtual": {
              "type": "keyword",
              "fields": {
                "fulltext": {
                  "type": "text"
                },
                "tree": {
                  "type": "text",
                  "analyzer": "fscrawler_path",
                  "fielddata": true
                }
              }
            }
          }
        }
      }
    }
  }
}

我可以使用标准 _search 从其他任何地方很好地搜索两个存储库，匹配“内容”对象

What is the correct <object> to use to connect my elasticsearch repository to the INCEpTION NER annotation program 

{
  "metadata": {
    "language": "en",
    "source": "My favourite document collection",
    "timestamp": "2011/11/11 11:11",
    "uri": "http://the.internet.com/my/document/collection/document1.txt",
    "title": "Cool Document Title"
  },
  "doc": {
      "text": "This is a test Document"
  }
}

即使将示例 1 向上移动，相同的查询也适用于示例 = 

{
  "metadata": {
    "language": "en",
    "source": "My favourite document collection",
    "timestamp": "2011/11/11 11:11",
    "uri": "http://the.internet.com/my/document/collection/document1.txt",
    "title": "Cool Document Title"
  },
  "doc": "This is a test Document"
  }
}

我需要在下面指定哪个对象才能访问我的映射的“内容”对象

What is the correct <object> to use to connect my elasticsearch repository to the INCEpTION NER annotation program

克里斯


【解决方案1】：

很遗憾，这是一个功能问题。

映射必须非常具体才能使用它，除非文档映射非常具体，否则重新映射（更改 fscrawler 映射）不起作用。

仅仅改变字段和类型是行不通的。

https://github.com/inception-project/inception/issues/2516

推荐
标签
  • go-gorm(18)
  • jwt(17)
  • visual-studio(26)
  • android(19)
  • react-native(20)
  • macos(32)
  • ajax(24)
  • woocommerce(19)
  • angular(431)
  • linux(31)
  • amazon-web-services(37)
  • elastic-stack(26)
  • azure(23)
  • selenium-webdriver(17)
  • dataframe(27)
  • .net-core(25)
  • ionic-framework(29)
  • date(17)
  • next.js(26)
  • sass(24)
  • unity3d(27)
  • vite(19)
  • angular-material(29)
  • 问答(4488)
  • flask(17)
  • database(69)
  • selenium(29)
  • go(392)
  • .net-6.0(22)
  • regex(21)
  • struct(16)
  • node.js(640)
  • string(21)
  • api(47)
  • elasticsearch(342)
  • c(85)
  • logstash(20)
  • discord.js(21)
  • firebase(32)
  • vuetify.js(26)
  • spring-data-elasticsearch(19)
  • java(89)
  • mysql(433)
  • sql(141)
  • kibana(34)
  • vue.js(374)
  • node-modules(17)
  • html(213)
  • electron-builder(46)
  • nuxt.js(29)
  • python(514)
  • css(107)
  • unit-testing(34)
  • python-3.x(52)
  • electron(306)
  • symfony(18)
  • arrays(81)
  • sorting(18)
  • javascript(961)
  • vuejs2(39)
  • jestjs(35)
  • pandas(68)
  • vuejs3(60)
  • matplotlib(16)
  • electron-forge(19)
  • typescript(269)
  • list(16)
  • rest(16)
  • postgresql(40)
  • c#(343)
  • django(23)
  • json(77)
  • numpy(25)
  • c++(154)
  • wordpress(41)
  • async-await(21)
  • .net(72)
  • laravel(125)
  • asp.net-core(42)
  • graphql(18)
  • reactjs(239)
  • kubernetes(22)
  • express(84)
  • visual-studio-code(44)
  • web-scraping(17)
  • php(457)
  • mongodb(69)
  • mongoose(33)
  • laravel-8(16)
  • spring-boot(39)
  • npm(71)
  • axios(39)
  • redis(21)
  • windows(23)
  • rxjs(34)
  • jquery(44)
  • docker(46)
  • webpack(36)
  • asp.net(40)
  • nestjs(18)
在线工具