分享知识,分享快乐

0%

solr command

生成本地配置

1
solrctl instancedir --generate $HOME/test_collection_config

上传到zk

1
solrctl instancedir --create test_collection_config $HOME/test_collection_config

创建collection

1
solrctl collection --create test_collection -s 1 -c test_collection_config

4. post数据

1
2
cd /opt/cloudera/parcels/CDH/share/doc/solr-doc*/example/exampledocs
java -Durl=http://bigdata-3.baofoo.cn:8983/solr/test_collection/update -jar post.jar *.xml

查看 hdfs dir

1
2
3
4
5
6
hadoop fs -ls -R /solr/test_co*/

drwxr-xr-x - solr solr 0 2019-05-08 22:07 /solr/test_collection/core_node2
drwxr-xr-x - solr solr 0 2019-05-08 22:07 /solr/test_collection/core_node2/data
drwxr-xr-x - solr solr 0 2019-05-08 22:12 /solr/test_collection/core_node2/data/index
-rwxr-xr-x 3 solr solr 82 2019-05-08 22:12 /solr/test_collection/core_node2/data/index/_0.dii

查看 zk dir

1
/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/zookeeper/bin/zkCli.sh -server localhost:2181

配置 config

Manager Configs

The solrctl config command syntax is as follows:

1
2
solrctl config [--create <name> <baseConfig> [-p <name>=<value>]...]
[--delete <name>]
  • –create : Creates a new config based on an existing config. The config is created with the specified , using as the template. For more information about config templates, see Config Templates.

  • -p =: Overrides a setting. The only config property that you can override is immutable, so the possible options are -p immutable=true and -p immutable=false. If you are copying an immutable config, such as a template, use -p immutable=false to make sure that you can edit the new config.

  • –delete : Deletes the specified config. You cannot delete an immutable config without accessing ZooKeeper directly as the solr super user.

  • example:

1
solrctl config --create newConfig test_collection_config -p immutable=false

Managing Instance Directories

  • An instance directory is a named set of configuration files. You can generate an instance directory template locally, edit the configuration, and then upload the directory to ZooKeeper as a named configuration set. You can then reference this named configuration set when creating a collection.

The solrctl instancedir command syntax is as follows:

1
2
3
4
5
6
solrctl instancedir [--generate <path> [-schemaless]]
[--create <name> <path>]
[--update <name> <path>]
[--get <name> <path>]
[--delete <name>]
[--list]
  • –generate : Generates an instance directory template on the local filesystem at . The configuration files are located in the conf subdirectory under .
  • -schemaless: Generates a schemaless instance directory template. For more information on schemaless support, see Schemaless Mode Overview and Best Practices.
  • –create : Uploads a copy of the instance directory from on the local filesystem to ZooKeeper. If an instance directory with the specified already exists, this command fails. Use --update to modify existing instance directories.
  • –update : Overwrites an existing instance directory in ZooKeeper using the specified files on the local filesystem. This command is analogous to first running --delete followed by --create .
  • –get : Downloads the specified instance directory from ZooKeeper to the specified path on the local filesystem. You can then edit the configuration and then re-upload it using --update.
  • –delete : Deletes the specified instance directory from ZooKeeper.
  • –list: Lists existing instance directories as well as configs created by the solrctl config command.
1
2
3
4
5
6
solrctl instancedir --get test_collection_config  /tmp/test_collection_config

cd /tmp/test_collection_config/conf/
vim managed-schema

solrctl instancedir --update test_collection_config /tmp/test_collection_config

schema api

  • Add a New Field
1
2
3
4
5
6
curl -X POST -H 'Content-type:application/json' --data-binary '{
"add-field":{
"name":"sell-by",
"type":"pdate",
"stored":true }
}' http://localhost:8983/solr/gettingstarted/schema
  • Delete a Field
1
2
3
curl -X POST -H 'Content-type:application/json' --data-binary '{
"delete-field" : { "name":"sell-by" }
}' http://localhost:8983/solr/gettingstarted/schema
  • Replace a Field
1
2
3
4
5
6
curl -X POST -H 'Content-type:application/json' --data-binary '{
"replace-field":{
"name":"sell-by",
"type":"date",
"stored":false }
}' http://localhost:8983/solr/gettingstarted/schema
  • Add a Dynamic Field Rule
1
2
3
4
5
6
curl -X POST -H 'Content-type:application/json' --data-binary '{
"add-dynamic-field":{
"name":"*_s",
"type":"string",
"stored":true }
}' http://localhost:8983/solr/gettingstarted/schema
  • Multiple Commands in a Single POST
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
curl -X POST -H 'Content-type:application/json' --data-binary '{
"add-field-type":{
"name":"myNewTxtField",
"class":"solr.TextField",
"positionIncrementGap":"100",
"analyzer":{
"charFilters":[{
"class":"solr.PatternReplaceCharFilterFactory",
"replacement":"$1$1",
"pattern":"([a-zA-Z])\\\\1+" }],
"tokenizer":{
"class":"solr.WhitespaceTokenizerFactory" },
"filters":[{
"class":"solr.WordDelimiterFilterFactory",
"preserveOriginal":"0" }]}},
"add-field" : {
"name":"sell-by",
"type":"myNewTxtField",
"stored":true }
}' http://localhost:8983/solr/gettingstarted/schema
  • Get the entire schema in JSON.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
curl http://localhost:8983/solr/gettingstarted/schema

{
"responseHeader":{
"status":0,
"QTime":5},
"schema":{
"name":"example",
"version":1.5,
"uniqueKey":"id",
"fieldTypes":[{
"name":"alphaOnlySort",
"class":"solr.TextField",
"sortMissingLast":true,
"omitNorms":true,
"analyzer":{
"tokenizer":{
"class":"solr.KeywordTokenizerFactory"},
"filters":[{
"class":"solr.LowerCaseFilterFactory"},
{
"class":"solr.TrimFilterFactory"},
{
"class":"solr.PatternReplaceFilterFactory",
"replace":"all",
"replacement":"",
"pattern":"([^a-z])"}]}}],
"fields":[{
"name":"_version_",
"type":"long",
"indexed":true,
"stored":true},
{
"name":"author",
"type":"text_general",
"indexed":true,
"stored":true},
{
"name":"cat",
"type":"string",
"multiValued":true,
"indexed":true,
"stored":true}],
"copyFields":[{
"source":"author",
"dest":"text"},
{
"source":"cat",
"dest":"text"},
{
"source":"content",
"dest":"text"},
{
"source":"author",
"dest":"author_s"}]}}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
curl http://localhost:8983/solr/gettingstarted/schema?wt=xml

<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">5</int>
</lst>
<lst name="schema">
<str name="name">example</str>
<float name="version">1.5</float>
<str name="uniqueKey">id</str>
<arr name="fieldTypes">
<lst>
<str name="name">alphaOnlySort</str>
<str name="class">solr.TextField</str>
<bool name="sortMissingLast">true</bool>
<bool name="omitNorms">true</bool>
<lst name="analyzer">
<lst name="tokenizer">
<str name="class">solr.KeywordTokenizerFactory</str>
</lst>
<arr name="filters">
<lst>
<str name="class">solr.LowerCaseFilterFactory</str>
</lst>
<lst>
<str name="class">solr.TrimFilterFactory</str>
</lst>
<lst>
<str name="class">solr.PatternReplaceFilterFactory</str>
<str name="replace">all</str>
<str name="replacement"/>
<str name="pattern">([^a-z])</str>
</lst>
</arr>
</lst>
</lst>
...
<lst>
<str name="source">author</str>
<str name="dest">author_s</str>
</lst>
</arr>
</lst>
</response>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
curl http://localhost:8983/solr/gettingstarted/schema?wt=schema.xml

<schema name="example" version="1.5">
<uniqueKey>id</uniqueKey>
<types>
<fieldType name="alphaOnlySort" class="solr.TextField" sortMissingLast="true" omitNorms="true">
<analyzer>
<tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.TrimFilterFactory"/>
<filter class="solr.PatternReplaceFilterFactory" replace="all" replacement="" pattern="([^a-z])"/>
</analyzer>
</fieldType>
...
<copyField source="url" dest="text"/>
<copyField source="price" dest="price_c"/>
<copyField source="author" dest="author_s"/>
</schema>
  • Get a list of all fields.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
curl http://localhost:8983/solr/gettingstarted/schema/fields

{
"fields": [
{
"indexed": true,
"name": "_version_",
"stored": true,
"type": "long"
},
{
"indexed": true,
"name": "author",
"stored": true,
"type": "text_general"
},
{
"indexed": true,
"multiValued": true,
"name": "cat",
"stored": true,
"type": "string"
},
"..."
],
"responseHeader": {
"QTime": 1,
"status": 0
}
}
  • Get a list of all dynamic field declarations:
1
curl http://localhost:8983/solr/gettingstarted/schema/dynamicfields
  • Get a list of all field types.
1
curl http://localhost:8983/solr/gettingstarted/schema/fieldtypes

引用参考

https://www.cloudera.com/documentation/enterprise/6/6.0/topics/search_validate_deploy_solr_rest_api.html

https://www.cloudera.com/documentation/enterprise/6/6.0/topics/search_configuration.html

https://lucene.apache.org/solr/guide/7_0/schema-api.html

管理页面

http://bigdata-3.baofoo.cn:8983/solr/#/~collections

http://bigdata-3.baofoo.cn:8889/hue/dashboard/browse/test_collection