{"id":2906,"date":"2021-05-07T14:39:00","date_gmt":"2021-05-07T02:39:00","guid":{"rendered":"https:\/\/www.talkcrypto.org\/blog\/?p=2906"},"modified":"2023-11-28T01:23:54","modified_gmt":"2023-11-27T12:23:54","slug":"part-2-understanding-subql-hello-world","status":"publish","type":"post","link":"https:\/\/www.talkcrypto.org\/blog\/2021\/05\/07\/part-2-understanding-subql-hello-world\/","title":{"rendered":"Part 2: Understanding SubQuery (aka Subql) Hello World in more depth"},"content":{"rendered":"\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Introduction<\/h2>\n\n\n\n<p>The article follows from <a href=\"https:\/\/www.talkcrypto.org\/blog\/2021\/06\/06\/part-1-subql-hello-world-in-less-than-5-mins\/\" target=\"_blank\" rel=\"noreferrer noopener\">Part 1: Subql Hello World in less than 5 mins<\/a> where we look under the covers at what is really going on because let&#8217;s face it. You can&#8217;t really understand anything new in 5 minutes. The exercise was really to get familiar with the technology, test connectivity and see if you could get something tangible working. <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Subql init<\/h2>\n\n\n\n<p>The first command we ran was:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>subql init --starter subqlHelloWorld<\/code><\/pre>\n\n\n\n<p>This does the heavy lifting and creates a whole bunch of files for you. As noted in the <a rel=\"noreferrer noopener\" href=\"https:\/\/doc.subquery.network\/quickstart.html#initialise-the-starter-subquery-project\" target=\"_blank\">official documentation<\/a>, &#8220;you will mainly be working on the following files:&#8221;<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>The Manifest in&nbsp;<code>project.yaml<\/code><\/li><li>The GraphQL Schema in&nbsp;<code>schema.graphql<\/code><\/li><li>The Mapping functions in&nbsp;<code>src\/mappings\/<\/code>&nbsp;directory<\/li><\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img fetchpriority=\"high\" decoding=\"async\" width=\"486\" height=\"642\" src=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/Screen-Shot-2021-06-06-at-9.35.39-am.png\" alt=\"\" class=\"wp-image-2908\" srcset=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/Screen-Shot-2021-06-06-at-9.35.39-am.png 486w, https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/Screen-Shot-2021-06-06-at-9.35.39-am-227x300.png 227w\" sizes=\"(max-width: 486px) 100vw, 486px\" \/><\/figure>\n\n\n\n<p>These files are the core of everything we do. As such, we&#8217;ll dedicate more time to these files in another article. For now though, just know that these files define the graphql schema, ie a blue print or a glossary of how you can query the blockchain, the project yaml file which contains &#8220;configuration&#8221; type parameters and of course the mappingHandlers typescript which can be seen as the code engine of everything. <\/p>\n\n\n\n<p><\/p>\n\n\n\n<p>The next thing we did was:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>yarn install<\/code><\/pre>\n\n\n\n<p>npm install can also be used as well. <\/p>\n\n\n\n<p>Short history lesson. Node Package Manager or npm was initially released in 2010 and is a tremendously popular package manager among JavaScript developers. It is the default package that is automatically installed whenever you install Node.js on your system. Yarn was initially released by Facebook in 2016 with the intention to address some of the performance and security shortcomings of working with npm (at that time). Feel free to Google for even more history. <\/p>\n\n\n\n<p>What yarn does is look at the package.json file and download various other dependencies. Looking at the package.json file, it doesn&#8217;t look like there are many dependencies, but when you run the command, you&#8217;ll notice that 18,983 files are added. This is because each dependency will also have their own dependencies. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-14.png\" alt=\"\" class=\"wp-image-2909\" width=\"595\" height=\"464\" srcset=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-14.png 618w, https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-14-300x234.png 300w\" sizes=\"(max-width: 595px) 100vw, 595px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>The next command we ran was:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>yarn codegen<\/code><\/pre>\n\n\n\n<p>What this does is it fetches the graphql schema and generates the associated typescript. (Hence the output have a .ts extension).<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"225\" height=\"149\" src=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-16.png\" alt=\"\" class=\"wp-image-2911\"\/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-15.png\" alt=\"\" class=\"wp-image-2910\" width=\"468\" height=\"235\" srcset=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-15.png 570w, https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-15-300x151.png 300w\" sizes=\"(max-width: 468px) 100vw, 468px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>Next the build command was run.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>yarn build<\/code><\/pre>\n\n\n\n<p>This will be familiar for seasoned programmers. This creates a dist folder performing things such as code optimisation preparing for a deployment. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"271\" height=\"303\" src=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-17.png\" alt=\"\" class=\"wp-image-2912\" srcset=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-17.png 271w, https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-17-268x300.png 268w\" sizes=\"(max-width: 271px) 100vw, 271px\" \/><\/figure>\n\n\n\n<p>The final step was the combined docker command. <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>docker-compose pull &amp;&amp; docker-compose up<\/code><\/pre>\n\n\n\n<p>This can be run separately as well. The <strong>pull<\/strong> command grabs all the required images from Docker Hub and the <strong>up<\/strong> command starts the container. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"702\" height=\"124\" src=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-18.png\" alt=\"\" class=\"wp-image-2913\" srcset=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-18.png 702w, https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-18-300x53.png 300w\" sizes=\"(max-width: 702px) 100vw, 702px\" \/><\/figure>\n\n\n\n<p>When the container is started, you&#8217;ll see the terminal spit out lots of text showing the status of the node and the graphql engine. It&#8217;s when you see: <\/p>\n\n\n\n<p>subquery-node_1 &nbsp; | 2021-06-06T02:04:25.490Z &lt;fetch&gt; INFO fetch block [1, 100]&nbsp;<\/p>\n\n\n\n<p>That you know that the subql node has started to synchronise. <\/p>\n\n\n\n<p>You may get errors such as: <\/p>\n\n\n\n<p>subquery-node_1 &nbsp; | 2021-06-06 02:05:34&nbsp; &nbsp; &nbsp; &nbsp; RPC-CORE: health(): Health:: disconnected from wss:\/\/polkadot.api.onfinality.io\/public-ws: 1006:: Connection dropped by remote peer.<\/p>\n\n\n\n<p>subquery-node_1 &nbsp; | 2021-06-06 02:05:34&nbsp; &nbsp; &nbsp; &nbsp; API\/INIT: Health keepalive check failed: disconnected from wss:\/\/polkadot.api.onfinality.io\/public-ws: 1006:: Connect<\/p>\n\n\n\n<p>If you do, stop the node with ctrl+c and start the container again. <\/p>\n\n\n\n<p>Then head over to http:\/\/localhost:3000\/ and run the following query<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>{\n  query{\n    starterEntities(last:10, orderBy: FIELD1_ASC){\n      nodes{\n        field1\n      }\n    }\n  }\n}<\/code><\/pre>\n\n\n\n<p>You should see:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>{\n  \"data\": {\n    \"query\": {\n      \"starterEntities\": {\n        \"nodes\": &#91;\n          {\n            \"field1\": 2046\n          },\n          {\n            \"field1\": 2047\n          },\n          {\n            \"field1\": 2048\n          },\n          {\n            \"field1\": 2049\n          },\n          {\n            \"field1\": 2050\n          },\n          {\n            \"field1\": 2051\n          },\n          {\n            \"field1\": 2052\n          },\n          {\n            \"field1\": 2053\n          },\n          {\n            \"field1\": 2054\n          },\n          {\n            \"field1\": 2055\n          }\n        ]\n      }\n    }\n  }\n}<\/code><\/pre>\n\n\n\n<p>This is returning the last 10 block count on the Polkadot mainnet and if you keep refreshing it by clicking the grey play button, you should see the block count change. <\/p>\n\n\n\n<p>Now depending on the speed of your computer and internet connection, it could take days for a full blockchain synchronisation. For me, the estimate time was:<\/p>\n\n\n\n<p>subquery-node_1 &nbsp; | 2021-06-06T02:09:49.285Z &lt;benchmark&gt; INFO 28.14 bps, target: #5381789, current: #5111, estimate time: 2 days 05 hours 04 mins&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Changing the starting block<\/h2>\n\n\n\n<p>A trick you can use is to modify the startBlock to something closer to the current block height which can be viewed at: https:\/\/polkascan.io\/ At the time of this article it was 5,381,818. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"242\" height=\"185\" src=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-19.png\" alt=\"\" class=\"wp-image-2915\"\/><\/figure>\n\n\n\n<p>I set the startBlock to 5.3M, <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"932\" height=\"568\" src=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/Screen-Shot-2021-06-06-at-12.32.01-pm.png\" alt=\"\" class=\"wp-image-2917\" srcset=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/Screen-Shot-2021-06-06-at-12.32.01-pm.png 932w, https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/Screen-Shot-2021-06-06-at-12.32.01-pm-300x183.png 300w, https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/Screen-Shot-2021-06-06-at-12.32.01-pm-768x468.png 768w\" sizes=\"(max-width: 932px) 100vw, 932px\" \/><\/figure>\n\n\n\n<p>Don&#8217;t forget to delete the hidden data folder as well if you are NOT running this for the first time. This is because this folder will have old block data from your previous startBlock number and if you don&#8217;t delete this, subql will try to restart from where it left off and not from the new startBlock value. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"433\" height=\"248\" src=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/Screen-Shot-2021-06-06-at-12.35.29-pm.png\" alt=\"\" class=\"wp-image-2918\" srcset=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/Screen-Shot-2021-06-06-at-12.35.29-pm.png 433w, https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/Screen-Shot-2021-06-06-at-12.35.29-pm-300x172.png 300w\" sizes=\"(max-width: 433px) 100vw, 433px\" \/><\/figure>\n\n\n\n<p>I then re-deployed it with docker-compose up and my block count started from 5.3M. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"638\" height=\"109\" src=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-20.png\" alt=\"\" class=\"wp-image-2916\" srcset=\"https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-20.png 638w, https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/image-20-300x51.png 300w\" sizes=\"(max-width: 638px) 100vw, 638px\" \/><\/figure>\n\n\n\n<p>This would then take only just over 1 hr to complete the sync. <\/p>\n\n\n\n<p>subquery-node_1 &nbsp; | 2021-06-06T02:33:33.995Z &lt;benchmark&gt; INFO 17.84 bps, target: #5382027, current: #5300497, estimate time: 0 days 01 hours 16 mins&nbsp;<\/p>\n\n\n\n<p>Of course, if you were doing historical queries older than 5.3M blocks, you would not get any data!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Summary<\/h2>\n\n\n\n<p>In this article we took a closer look at what the various commands we ran blindly in <a href=\"https:\/\/www.talkcrypto.org\/blog\/2021\/06\/06\/part-1-subql-hello-world-in-less-than-5-mins\/\" target=\"_blank\" rel=\"noreferrer noopener\">Part 1: Subql Hello World in less than 5 mins<\/a> actually meant. In subsequent articles we&#8217;ll take a closer look at how we can pull out or &#8220;query&#8221; for more useful information. See you then!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction The article follows from Part 1: Subql Hello World<\/p>\n","protected":false},"author":1,"featured_media":3423,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-2906","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blockchain"],"featured_image_urls":{"full":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery.png",1800,773,false],"thumbnail":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-150x150.png",150,150,true],"medium":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-300x129.png",300,129,true],"medium_large":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-768x330.png",640,275,true],"large":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-1024x440.png",640,275,true],"1536x1536":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-1536x660.png",1536,660,true],"2048x2048":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery.png",1800,773,false],"chromenews-featured":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-1024x440.png",1024,440,true],"chromenews-large":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-825x575.png",825,575,true],"chromenews-medium":["https:\/\/www.talkcrypto.org\/blog\/wp-content\/uploads\/2021\/06\/subquery-590x410.png",590,410,true]},"author_info":{"info":["seandotau"]},"category_info":"<a href=\"https:\/\/www.talkcrypto.org\/blog\/category\/blockchain\/\" rel=\"category tag\">Blockchain<\/a>","tag_info":"Blockchain","comment_count":"0","_links":{"self":[{"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/posts\/2906","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/comments?post=2906"}],"version-history":[{"count":9,"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/posts\/2906\/revisions"}],"predecessor-version":[{"id":2988,"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/posts\/2906\/revisions\/2988"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/media\/3423"}],"wp:attachment":[{"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/media?parent=2906"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/categories?post=2906"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.talkcrypto.org\/blog\/wp-json\/wp\/v2\/tags?post=2906"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}