{"id":3051,"date":"2021-06-26T00:42:00","date_gmt":"2021-06-25T12:42:00","guid":{"rendered":"https:\/\/www.talkcrypto.org\/blog\/?p=3051"},"modified":"2023-11-28T01:22:55","modified_gmt":"2023-11-27T12:22:55","slug":"how-to-run-a-subquery-indexer","status":"publish","type":"post","link":"https:\/\/www.talkcrypto.org\/blog\/2021\/06\/26\/how-to-run-a-subquery-indexer\/","title":{"rendered":"How to run a SubQuery Indexer?"},"content":{"rendered":"\n

<\/p>\n\n\n\n

At first, it may sound confusing but running an indexer is synonymous to running a node which will “index” the blockchain and store the data in a database. To be more precise, the @subql\/node implementation will extract substrate-based blockchain data (aka on the PolkaDot network) and save it into a Postgres database. <\/p>\n\n\n\n

Currently you don’t “join” a network. You create a SubQuery project and “run” it.<\/p>\n\n\n\n

Postgres<\/h2>\n\n\n\n

The first thing you will need to do is to install Postgres. Visit https:\/\/www.postgresql.org\/download\/<\/a> and install version 12 or higher.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

SubQuery node<\/h2>\n\n\n\n

To install the SubQuery node application, run<\/p>\n\n\n\n

npm install -g @subql\/node<\/code><\/pre>\n\n\n\n
The -g flag means to install it globally which means on OSX, the location will be \/usr\/local\/lib\/node_modules<\/p>\n\n\n\n
Once installed, check out the possible flags with the help command.<\/p>\n\n\n\n
> subql-node --help\nOptions:\n      --help                Show help                                  [boolean]\n      --version             Show version number                        [boolean]\n  -f, --subquery            Local path of the subquery project          [string]\n      --subquery-name       Name of the subquery project                [string]\n  -c, --config              Specify configuration file                  [string]\n      --local               Use local mode                             [boolean]\n      --batch-size          Batch size of blocks to fetch in one round  [number]\n      --timeout             Timeout for indexer sandbox to execute the mapping\n                            functions                                   [number]\n      --debug               Show debug information to console output. will\n                            forcefully set log level to debug\n                                                      [boolean] [default: false]\n      --profiler            Show profiler information to console output\n                                                      [boolean] [default: false]\n      --network-endpoint    Blockchain network endpoint to connect      [string]\n      --output-fmt          Print log as json or plain text\n                                           [string] [choices: \"json\", \"colored\"]\n      --log-level           Specify log level to print. Ignored when --debug is\n                            used\n          [string] [choices: \"fatal\", \"error\", \"warn\", \"info\", \"debug\", \"trace\",\n                                                                       \"silent\"]\n      --migrate             Migrate db schema (for management tables only)\n                                                      [boolean] [default: false]\n      --timestamp-field     Enable\/disable created_at and updated_at in schema\n                                                       [boolean] [default: true]\n  -d, --network-dictionary  Specify the dictionary api for this network [string]\n<\/code><\/pre>\n\n\n\n
#Connecting to a database<\/h2>\n\n\n\n
Use the export command to set the environment variables as shown below. <\/p>\n\n\n\n
export DB_USER=postgres\nexport DB_PASS=postgres\nexport DB_DATABASE=postgres\nexport DB_HOST=localhost\nexport DB_PORT=5432<\/code><\/pre>\n\n\n\n
Typing “env” should list your current environment variables and note that these variables are only set temporarily. In other words, they are only valid for the duration that the terminal window is open for. You can make it more permanent by creating these in your .\/bash_profile. <\/p>\n\n\n\n
Starting a project<\/h2>\n\n\n\n
To start a project, use the -f flag.<\/p>\n\n\n\n
subql-node -f <project_path><\/code><\/pre>\n\n\n\n
For example,<\/p>\n\n\n\n
subql-node -f . \/\/for current directory\nsubql-node -f .\/helloworld \/\/ for helloworld directory \n\n2021-08-09T23:24:08.778Z <fetch> INFO fetch block [6401,6500], total 100 blocks \n2021-08-09T23:24:11.807Z <fetch> INFO fetch block [6501,6600], total 100 blocks \n<\/code><\/pre>\n\n\n\n
Postgres<\/h2>\n\n\n\n
If you open up Postgres, you’ll see that two tables are created. public.subqueries and subquery_1.starter_entities<\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
The starter_entities table contains the indexes. To view the data, run:<\/p>\n\n\n\n
select * from subquery_1.starter_entities<\/code><\/pre>\n\n\n\n
\"\"<\/figure>\n\n\n\n
public.subqueries only contains 1 row which the node checks upon start up to “understand the current state” so it knows where to continue from so to speak. <\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
<\/p>\n\n\n\n
SubQuery query service<\/h1>\n\n\n\n
Now that we have the node running, the next step is to get the query service up and running. Run:<\/p>\n\n\n\n
npm install -g @subql\/query<\/code><\/pre>\n\n\n\n
To start the service, run:<\/p>\n\n\n\n
subql-query --name subql-helloworld --playground<\/code><\/pre>\n\n\n\n
If you get this error message, “Error: SASL: SCRAM-SERVER-FIRST-MESSAGE: client password must be a string”, this means that you need to set your DB_PASS again. This could be the result of you opening up another terminal meaning the environment variables need to be set again. <\/p>\n\n\n\n
Also make sure to set your DB_USER via:<\/p>\n\n\n\n
export DB_USER=postgres<\/code><\/pre>\n\n\n\n
The next error you will probably encounter is “Error: listen EADDRINUSE: address already in use :::3000”. <\/p>\n\n\n\n
Set a different PORT variable via:<\/p>\n\n\n\n
export PORT=3001<\/code><\/pre>\n\n\n\n
You should then see:<\/p>\n\n\n\n
~\/Code\/subQuery\/projects\/subql-helloworld$ export PORT=3002\n~\/Code\/subQuery\/projects\/subql-helloworld$ subql-query --name subql-helloworld --playground\n(node:20940) [PINODEP007] Warning: bindings.level is deprecated, use options.level option instead\n(Use `node --trace-warnings ...` to show where the warning was created)\n2021-08-14T02:29:58.525Z <nestjs> INFO Starting Nest application... \n2021-08-14T02:29:58.534Z <nestjs> INFO AppModule dependencies initialized \n2021-08-14T02:29:58.534Z <nestjs> INFO ConfigureModule dependencies initialized \n2021-08-14T02:29:58.534Z <nestjs> INFO GraphqlModule dependencies initialized \n\u26a0\ufe0f WARNING\u26a0\ufe0f  You requested to use schema 'subquery_1'; however we couldn't find some of those! Missing schemas are: 'subquery_1'\n2021-08-14T02:29:58.752Z <nestjs> INFO Nest application successfully started \n(node:20940) [PINODEP007] Warning: bindings.level is deprecated, use options.level option instead<\/code><\/pre>\n\n\n\n
Then navigate to http:\/\/localhost:3002\/ and you should see….<\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
Then run your query as normal.<\/p>\n","protected":false},"excerpt":{"rendered":"
At first, it may sound confusing but running an indexer<\/p>\n","protected":false},"author":1,"featured_media":3423,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-3051","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blockchain"],"featured_image_urls":{"full":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery.png",1800,773,false],"thumbnail":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-150x150.png",150,150,true],"medium":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-300x129.png",300,129,true],"medium_large":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-768x330.png",640,275,true],"large":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-1024x440.png",640,275,true],"1536x1536":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-1536x660.png",1536,660,true],"2048x2048":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery.png",1800,773,false],"chromenews-featured":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-1024x440.png",1024,440,true],"chromenews-large":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-825x575.png",825,575,true],"chromenews-medium":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-590x410.png",590,410,true]},"author_info":{"info":["seandotau"]},"category_info":"Blockchain<\/a>","tag_info":"Blockchain","comment_count":"0","_links":{"self":[{"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/posts\/3051","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/comments?post=3051"}],"version-history":[{"count":6,"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/posts\/3051\/revisions"}],"predecessor-version":[{"id":3097,"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/posts\/3051\/revisions\/3097"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/media\/3423"}],"wp:attachment":[{"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/media?parent=3051"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/categories?post=3051"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/tags?post=3051"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}