CURRICULUM
Section 0: Pre Intro
Section 1: Analysis
Section 2: Taxation and Regulation
Section 3: Last months in crypto market
Section 4: Algo crack
Section 5: AI crypto
Section 6: Live to Mine
Almost every developer in the world has heard of the blockchain but most still don’t know how it works. They might only know about it because of Bitcoin and because they’ve heard of things like smart contracts.
Let’s get started!
Setup
Since you’re going to write your code in Go, I assume you have had some experience with Go. After installing and configuring Go, you’ll also want to grab the following packages:
go get github.com/davecgh/go-spew/spew
Spew allows you to view structs and slices cleanly formatted in your console.
This is nice to have.
go get github.com/gorilla/mux
Gorilla/mux is a popular package for writing web handlers. You’ll need this.
go get github.com/joho/godotenv
Gotdotenv lets you read from a .env file that you keep in the root of your directory so you don’t have to hardcode things like your http ports. You’ll need this too.
Let’s also create a .env file in the root of your directory defining the port that will serve http requests. Just add one line to this file:
ADDR=8080
Create a main.go file. Everything from now on will be written to this. Let’s get coding!
Imports
Here are the imports you’ll need, along with your package declaration. Let’s write these to main.go
package main
import (
“crypto/sha256”
“encoding/hex”
“encoding/json”
“io”
“log”
“net/http”
“os”
“time”
“github.com/davecgh/go-spew/spew”
“github.com/gorilla/mux”
“github.com/joho/godotenv”
)
Data model
Let’s define the struct of each of your blocks that will make up the blockchain. Don’t worry, I’ll explain what all of these fields mean.
type Block struct {
Index int
Timestamp string
BPM int
Hash string
PrevHash string
}
Each Block contains data that will be written to the blockchain.
• Index is the position of the data record in the blockchain
• Timestamp is automatically determined and is the time the data is written
• BPM or beats per minute, is your rate
• Hash is a SHA256 identifier representing the data record
• PrevHash is the SHA256 identifier of the previous record in the chain
Let’s also model out the blockchain itself, which is simply a slice of Block:
var Blockchain [] Block
So how does hashing fit into blocks and the blockchain? We use hashes to identify and keep the blocks in the right order. By ensuring the PrevHash in each Block is identical to Hash in the previous Block we know the proper order of the blocks that make up the chain.
Hashing and Generating New Blocks
So why do we need hashing? We hash data for 2 main reasons:
• To save space. Hashes are derived from all the data that is on the block. In your case, you only have a few data points but imagine you would have data from hundreds, thousands or millions of previous blocks. It’s much more efficient to hash that data into a single SHA256 string or hash the hashes than to copy all the data in preceding blocks over and over again.
• Preserve integrity of the blockchain. By storing previous hashes like done in the diagram above, you’re able to ensure the blocks in the blockchain are in the right order. If a malicious party were to come in and try to manipulate the data, the hashes would change quickly and the chain would “break”, and everyone would know to not trust that malicious chain.
Let’s write a function that takes our Block data and creates a SHA256 hash of it.
func calculateHash(block Block) string {
record := string(block.Index) + block.Timestamp + string(block.BPM) + block.PrevHash
h := sha256.New()
h.Write([]byte(record))
hashed := h.Sum(nil)
return hex.EncodeToString(hashed)
}
This calculateHash function concatenates Index, Timestamp, BPM, PrevHash of the Block you provide as an argument and returns the SHA256 hash as a string. Now you can generate a new Block with all the elements you need with a new generateBlock function. You’ll need to supply it the previous block so you can get its hash and our rate in BPM. Don’t worry about the BPM int argument that’s passed in. You’ll address that later.
func generateBlock(oldBlock Block, BPM int) (Block, error) {
var newBlock Block
t := time.Now()
newBlock.Index = oldBlock.Index + 1
newBlock.Timestamp = t.String()
newBlock.BPM = BPM
newBlock.PrevHash = oldBlock.Hash
newBlock.Hash = calculateHash(newBlock)
return newBlock, nil
}
Notice that the current time is automatically written in the block with time.Now(). Also notice that your prior calculateHash function was called. PrevHash is copied over from the hash of the previous block. Index is incremented from the Index of the previous block.
Block Validation
Now you need to write some functions to make sure the blocks haven’t been tampered with. You do this by checking Index to make sure they’ve incremented as expected. You also check to make sure your PrevHash is indeed the same as the Hash of the previous block. Lastly, you want to double check the hash of the current block by running the calculateHash function again on the current block. Let’s write a isBlockValid function that does all these things and returns a bool. It’ll return true if it passes all your checks.
func isBlockValid(newBlock, oldBlock Block) bool {
if oldBlock.Index+1 != newBlock.Index {
return false
}
if oldBlock.Hash != newBlock.PrevHash {
return false
}
if calculateHash(newBlock) != newBlock.Hash {
return false
}
return true
}
What if you run into an issue where two nodes of your blockchain ecosystem both added blocks to their chains and you received them both. Which one do you pick as the source of truth? You choose the longest chain. This is a classic blockchain issue and has nothing to do with nefarious actors.
Two well meaning nodes may simply have different chain lengths, so naturally the longer one will be the most up to date and have the latest blocks. So let’s make sure the new chain we’re taking in is longer than the current chain you have. If it is, you can overwrite your chain with the new one that has the new block(s).
You simply compare the length of the slices of the chains to accomplish this.
func replaceChain(newBlocks []Block) {
if len(newBlocks) > len(Blockchain) {
Blockchain = newBlocks
}
}
You’ve basically written up the guts of your blockchain with all the various functions you need.
Now you want a convenient way to view your blockchain and write to it, ideally in a web browser so you can see it.
Web Server
I assume you’re already familiar with how web servers work and have a bit of experience wiring them up in Go. I’ll walk you through the process now.
You’ll be using the Gorilla/mux package that you downloaded earlier to do the heavy lifting.
Let’s create your server in a run function that you’ll call later.
func run() error {
mux := makeMuxRouter()
httpAddr := os.Getenv(“ADDR”)
log.Println(“Listening on “, os.Getenv(“ADDR”))
s := &http.Server{
Addr: “:” + httpAddr,
Handler: mux,
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
}
if err := s.ListenAndServe(); err != nil {
return err
}
return nil
}
Note that port you choose comes from your .env file that you created earlier. You give yourself a quick console message with log.Println to let you know the server is up and running. You configure the server a bit and then ListenAndServe.
Pretty standard stuff.
Now you need to write the makeMuxRouter function that will define all your handlers. To view and write to your blockchain in a browser, you only need 2 routes and you’ll keep them simple. If you send a GET request to localhost you’ll view your blockchain. If you send a POST request to it, you can write to it.
func makeMuxRouter() http.Handler {
muxRouter := mux.NewRouter()
muxRouter.HandleFunc(“/”, handleGetBlockchain).Methods(“GET”)
muxRouter.HandleFunc(“/”, handleWriteBlock).Methods(“POST”)
return muxRouter
}
Here’s our GET handler:
func handleGetBlockchain(w http.ResponseWriter, r *http.Request) {
bytes, err := json.MarshalIndent(Blockchain, “”, ” “)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
io.WriteString(w, string(bytes))
}
You simply write back the full blockchain, in JSON format, that you can view in any browser by visiting localhost:8080. You have ourADDR variable set as 8080 in your `.env` file so if you change it, make sure to visit your correct port.
Your POST request is a little bit more complicated, but not by much. To start, you’ll need a new Message struct. I’ll explain in a second why you need it.
type Message struct {
BPM int
}
Here’s the code for the handler that writes new blocks.
func handleWriteBlock(w http.ResponseWriter, r *http.Request) {
var m Message
decoder := json.NewDecoder(r.Body)
if err := decoder.Decode(&m); err != nil {
respondWithJSON(w, r, http.StatusBadRequest, r.Body)
return
}
defer r.Body.Close()
newBlock, err := generateBlock(Blockchain[len(Blockchain)-1], m.BPM)
if err != nil {
respondWithJSON(w, r, http.StatusInternalServerError, m)
return
}
if isBlockValid(newBlock, Blockchain[len(Blockchain)-1]) {
newBlockchain := append(Blockchain, newBlock)
replaceChain(newBlockchain)
spew.Dump(Blockchain)
}
respondWithJSON(w, r, http.StatusCreated, newBlock)
}
The reason you used a separate Message struct is to take in the request body of the JSON POST request you’ll use to write new blocks. This allows you to simply send a POST request with the following body and your handler will fill in the rest of the block for you:
{“BPM”:50}
The 50 is an example rate.
After you’re done decoding the request body into your var m Message struct, you create a new block by passing in the previous block and your new rate into the generateBlock function you wrote earlier . This is everything the function needs to create a new block. You do a quick check to make sure the new block is kosher using the isBlockValid function you created earlier.
A couple notes
• spew.Dump is a convenient function that pretty prints your structs into the console. It’s useful for debugging.
• for testing POST requests, use Postman. Curl works well too, if you just can’t get away from the terminal.
When your POST requests are successful or unsuccessful, you want to be alerted accordingly. You use a little wrapper function respondWithJSON to let you know what happened. Remember, in Go, never ignore errors.
func respondWithJSON(w http.ResponseWriter, r *http.Request, code int, payload interface{}) {
response, err := json.MarshalIndent(payload, “”, ” “)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
w.Write([]byte(“HTTP 500: Internal Server Error”))
return
}
w.WriteHeader(code)
w.Write(response)
}
Let’s wire all these different blockchain functions, web handlers and the web server in a short, clean main function:
func main() {
err := godotenv.Load()
if err != nil {
log.Fatal(err)
}
go func() {
t := time.Now()
genesisBlock := Block{0, t.String(), 0, “”, “”}
spew.Dump(genesisBlock)
Blockchain = append(Blockchain, genesisBlock)
}()
log.Fatal(run())
}
So what’s going on here?
• godotenv.Load() allows you to read in variables like your port number from the.env file you placed at the root of your directory so you don’t have to hardcode them throughout your app.
• genesisBlock is the most important part of the main function. You need to supply your blockchain with an initial block, or else a new block will not be able to compare its previous hash to anything, since a previous hash doesn’t exist.
• You isolate the genesis block into its own go routine so you can have a separation of concerns from your blockchain logic and your web server logic. This will work without the go routine but it’s just cleaner this way.
You’re done!
Now for the fun stuff. Let’s try it out.
Fire up your application from terminal using go run main.go
In terminal, you see that the web server is up and running and you get a printout of your genesis block.
Now, visit localhost with your port number, which for you is 8080. As expected, you see the same genesis block.
Now, let’s send some POST requests to add blocks. Using Postman, you’re going to add a new blocks with various BPMs.
Refresh your browser. Lo and behold, you now see all your new blocks in your chain with the PrevHash of your new ones matching the Hash of the old ones, just as you expected!
Next Steps
Congrats!! You just wrote up your own blockchain with proper hashing and block validation. Let’s pass over to networking.
I’ll show you here.
Workflow
• The first terminal creates a genesis block and a TCP server to which new nodes can connect
Step 1
• Additional terminals are opened and TCP connections are made to the first terminal
• New terminals write a block to the first terminal
Step 2
• The first terminal validates the block
• The first terminal broadcasts the new blockchain to each of the new nodes
Step 3
• All terminals now have synced blockchains!
You won’t be able to have computers in other networks writing to your first terminal, but this can be readily achieved by putting your binary in the cloud. Also, blockchain broadcasting will be simulated to each of the nodes. Don’t worry, I’ll explain all of this shortly.
Let’s get coding!
You’ll be using many of the same block generation, hashing and validation functions. But you won’t be using any of the HTTP functionality since you’ll be viewing results in the console and you’ll be using TCP for networking.
What’s the difference between TCP and HTTP?
I won’t get into much detail here but all you need to know is that TCP is a base protocol that transmits data. HTTP is built on top of TCP to utilize this data transmission on the web and in browsers. When you view a website, you’re using HTTP, which is supported by an underlying data transmission protocol, called TCP.
Go has a nice net package that provides all the TCP connection functions you need.
Setup, Imports and Review
For blockchain generation and validation, you’ll be using the same functions from before.
Setup
Create a .env file in your root directory and add the following line:
ADDR=9000
You store the TCP port number you want to use (in this case 9000) in an environment variable called ADDR.
If you haven’t done it already, grab the following packages:
go get github.com/davecgh/go-spew/spew to pretty print your blockchain to the console
go get github.com/joho/godotenv to load the variables in your .env file
Create an empty main.go file. You’ll be putting all your code here.
Imports
Do your standard package declaration and write up the imports you’ll need.
package main
import (
“bufio”
“crypto/sha256”
“encoding/hex”
“encoding/json”
“io”
“log”
“net”
“os”
“strconv”
“time”
“github.com/davecgh/go-spew/spew”
“github.com/joho/godotenv”
)
Review
Let’s create your Block struct and declare your Blockchain as a slice of Block
// Block represents each ‘item’ in the blockchain
type Block struct {
Index int
Timestamp string
BPM int
Hash string
PrevHash string
}
// Blockchain is a series of validated Blocks
var Blockchain []Block
Now create your hashing function that you need when generating new blocks.
// SHA256 hashing
func calculateHash(block Block) string {
record := string(block.Index) + block.Timestamp + string(block.BPM) + block.PrevHash
h := sha256.New()
h.Write([]byte(record))
hashed := h.Sum(nil)
return hex.EncodeToString(hashed)
}
Your block generation function:
// create a new block using previous block’s hash
func generateBlock(oldBlock Block, BPM int) (Block, error) {
var newBlock Block
t := time.Now()
newBlock.Index = oldBlock.Index + 1
newBlock.Timestamp = t.String()
newBlock.BPM = BPM
newBlock.PrevHash = oldBlock.Hash
newBlock.Hash = calculateHash(newBlock)
return newBlock, nil
}
Make sure your new block is valid by checking its PrevHashagainst the Hash of the previous block.
// make sure block is valid by checking index, and comparing the hash of the previous block
func isBlockValid(newBlock, oldBlock Block) bool {
if oldBlock.Index+1 != newBlock.Index {
return false
}
if oldBlock.Hash != newBlock.PrevHash {
return false
}
if calculateHash(newBlock) != newBlock.Hash {
return false
}
return true
}
Now you ensure you take the longest chain as the true blockchain:
// make sure the chain we’re checking is longer than the current blockchain
func replaceChain(newBlocks []Block) {
if len(newBlocks) > len(Blockchain) {
Blockchain = newBlocks
}
}
Nice! You’ve basically got all the blockchain related functions you need with all the HTTP related stuff from previous part stripped out. You can now proceed to networking.
Networking
Finally! You’re at the good stuff. Let’s set up a network that can pass around new blocks, integrate them into your blockchain, and broadcast new blockchains back to the network.
Let’s start with your main function, as that is a nice abstraction that will help you understand the overall flow.
Right before you do that, let’s declare a global variable called bcServer (short for blockchain server) under your other struct declarations that is a channel that takes in incoming blocks.
// bcServer handles incoming concurrent Blocks
var bcServer chan []Block
Side note: Channels are one of the most popular features in Go and allow for elegant streamlining of data reading/writing and are most often used to prevent data races. They become particularly powerful when used concurrently, as multiple Go routines can write to the same channel. Traditionally in Java and other C-like language, you’d have to open and close the “gates” for incoming data through mutexes. Channels in Go make this much easier, although there are still places for mutexes in Go.
Now let’s declare your main function and load up the environment variable from your .env file that sits in our root directory. Remember, the only environment variable is ADDR which is our TCP port number 9000 which you’ll use in a second. Also, let’s instantiate your bcServer in the main function.
func main() {
err := godotenv.Load()
if err != nil {
log.Fatal(err)
}
bcServer = make(chan []Block)
// create genesis block
t := time.Now()
genesisBlock := Block{0, t.String(), 0, “”, “”}
spew.Dump(genesisBlock)
Blockchain = append(Blockchain, genesisBlock)
}
Now you need to write up your TCP server. Remember, you can think of TCP servers as being similar to HTTP servers but there is no browser component. All data transmission will be done in the console through your terminal. You will be handling multiple connections to your TCP port. Add this to your main function below the last line.
// start TCP and serve TCP server
server, err := net.Listen(“tcp”, “:”+os.Getenv(“ADDR”))
if err != nil {
log.Fatal(err)
}
defer server.Close()
This fires up your TCP server at port 9000. It’s important to defer server.Close() so the connection closes cleanly when you no longer need it.
Now you need to create a new connection each time you receive a connection request, and you need to serve it. Add this below the last line.
for {
conn, err := server.Accept()
if err != nil {
log.Fatal(err)
}
go handleConn(conn)
}
You just made an infinite loop where you accept new connections. You want to concurrently deal with each connection through a separate handler in a Go routine go handleConn(conn), so you don’t clog up your for loop. This is how you can serve multiple connections concurrently.
You now will jump up and say, “hey wait! I don’t have a handleConn function!”. You’re right. But let’s catch a quick breath. Nice job, you just wrote up your entire main function. It looks like this:
func main() {
err := godotenv.Load()
if err != nil {
log.Fatal(err)
}
bcServer = make(chan []Block)
// create genesis block
t := time.Now()
genesisBlock := Block{0, t.String(), 0, “”, “”}
spew.Dump(genesisBlock)
Blockchain = append(Blockchain, genesisBlock)
// start TCP and serve TCP server
server, err := net.Listen(“tcp”, “:”+os.Getenv(“ADDR”))
if err != nil {
log.Fatal(err)
}
defer server.Close()
for {
conn, err := server.Accept()
if err != nil {
log.Fatal(err)
}
go handleConn(conn)
}
}
Let’s write up that handleConn function now. It only takes in one argument, a nicely packaged net.Conn interface. Interfaces in Go are amazing and in my opinion, what separates Go from every other C-based language. Concurrency and Go routines get all the hype but interfaces and the fact that they’re satisfied implicitly are the language’s most powerful feature. If you don’t use interfaces in Go yet, get familiar with them as quickly as you can. Interfaces are your next gen step as developer!
Put in the skeleton of the function and start it off with a clean defer statement to close each connection when you’re done with it.
func handleConn(conn net.Conn) {
defer conn.Close()
}
Now you need to allow the client to add new blocks for you to integrate into the blockchain. You’re going to use the same rate from before. That will be your BPM.
To achieve the above you need to:
• prompt the client to enter their BPM
• scan the client’s input from stdin
• create a new block with this data, using the generateBlock, isBlockValid, and replaceChain functions we created previously
• put the new blockchain in the channel we created to be broadcast to the network
• allow the client to enter a new BPM
Here is the code to do that, in the exact order described above.
io.WriteString(conn, “Enter a new BPM:”)
scanner := bufio.NewScanner(conn)
// take in BPM from stdin and add it to blockchain after conducting necessary validation
go func() {
for scanner.Scan() {
bpm, err := strconv.Atoi(scanner.Text())
if err != nil {
log.Printf(“%v not a number: %v”, scanner.Text(), err)
continue
}
newBlock, err := generateBlock(Blockchain[len(Blockchain)-1], bpm)
if err != nil {
log.Println(err)
continue
}
if isBlockValid(newBlock, Blockchain[len(Blockchain)-1]) {
newBlockchain := append(Blockchain, newBlock)
replaceChain(newBlockchain)
}
bcServer <- Blockchain
io.WriteString(conn, “\nEnter a new BPM:”)
}
}()
You create a new scanner. The for scanner.Scan() loop is tucked away in its own Go routine so it can run concurrently and separately from other connections. You do a quick string conversion of your BPM (which will always be an integer, so we check for that). You do your standard block generation and validation checks and create a new blockchain with the new block.
The syntax bcServer <- Blockchain just means you’re throwing your new blockchain into the channel you created. Then you prompt the client to enter a new BPM to create the next block.
Broadcasting
You need to broadcast the new blockchain to all the connections being served by your TCP server. Since you’re coding this up on one computer, you’ll be simulating how data gets transmitted to all clients. Under the last line of code, in the same handleConn function, you need to
• Marshall your new blockchain into JSON so you can read it nicely
• write the new blockchain to the consoles of each of your connections
• set a timer to do this periodically so you’re not getting swallowed with blockchain data. This is also what you see in live blockchain networks, where new blockchains are broadcast every X minutes. Use 30 seconds
• Pretty print the main blockchain to the first terminal, just so you can see what’s going on and ensure blocks being added by different nodes are indeed being integrated into the main blockchain
Here is the code, in this exact order.
// simulate receiving broadcast
go func() {
for {
time.Sleep(30 * time.Second)
output, err := json.Marshal(Blockchain)
if err != nil {
log.Fatal(err)
}
io.WriteString(conn, string(output))
}
}()
for _ = range bcServer {
spew.Dump(Blockchain)
}
Nice! Your handleConn function is done. In fact, the whole program is done.
Let’s try this out! Fire up your application by going to your directory and go run main.go
As expected, you see your genesis block. What’s happening at the same time is that you’ve started a TCP server at port 9000 that can take in multiple connections. So let’s do that.
Open a new terminal window and connect to your TCP server with nc localhost 9000. You’ll be using different color terminals to make it clear these are different clients. Do this a couple times with a couple terminal sessions to activate multiple clients.
Now type in a BPM to any of the clients. And TA-DA! You see the new block added to the first terminal! Networking in action!
Here’s where it gets REALLY cool. Wait 30 seconds. Go to one of the other clients and you’ll see the new blockchain broadcast to all the clients, even if those clients never inputted a BPM!
Next steps
Cool! This is all a blockchain network is. It needs to take in incoming data and also broadcast data externally. Doing both in a single terminal session is effectively what a mining setup is.
Created your own blockchain and added networking. Nice.
There are several places you can go from here:
• To have a bigger network running on your local environment, create multiple directories that hold a copy of the application each with different TCP ports. For each terminal session, serve a TCP port and connect to a different one so you can receive and send data.
• Connect data being streamed from multiple ports.
• If you want to try this, set up the server in the cloud using your favorite hosting provider. Have your friends connect to it and send in data. There are some additional security considerations here that you must go through.
Now onto some Proof of Work!
You’ll also add a calculateHash function that generates the hashes you need to create Hash and PrevHash. This is just a SHA-256 hash of our concatenated index, timestamp, BPM, previous hash and Nonce.
Proof of Work
Let’s get to the mining algorithm. You want to make sure the Proof of Work is complete before you’ll allow a new Block to get added to the blockchain. Let’s start off with a simple function that checks if the hash generated during your Proof of Work meets the requirements you set out.
Your requirements will be as follows:
• The hash that is generated by your Proof of Work must start with a specific number of zeros
• The number of zeros is determined by the constant difficulty that you defined at the start of your program (in your case, 1)
• You can make the Proof of Work harder to solve by increasing the difficulty
Write up this function, isHashValid:
func isHashValid(hash string, difficulty int) bool {
prefix := strings.Repeat(“0”, difficulty)
return strings.HasPrefix(hash, prefix)
}
Go provides convenient Repeat and HasPrefix functions in its strings package. You define the variable prefix as a repeat of zeros defined by your difficulty. Then you check if the hash starts with those zeros and return True if it does and False if it doesn’t.
Now let’s create your generateBlock function.
func generateBlock(oldBlock Block, BPM int) Block {
var newBlock Block
t := time.Now()
newBlock.Index = oldBlock.Index + 1
newBlock.Timestamp = t.String()
newBlock.BPM = BPM
newBlock.PrevHash = oldBlock.Hash
newBlock.Difficulty = difficulty
for i := 0; ; i++ {
hex := fmt.Sprintf(“%x”, i)
newBlock.Nonce = hex
if !isHashValid(calculateHash(newBlock), newBlock.Difficulty) {
fmt.Println(calculateHash(newBlock), ” do more work!”)
time.Sleep(time.Second)
continue
} else {
fmt.Println(calculateHash(newBlock), ” work done!”)
newBlock.Hash = calculateHash(newBlock)
break
}
}
return newBlock
}
You create a newBlock that takes the hash of the previous block and puts it in PrevHash to make sure you have continuity in our blockchain. Most of the other fields should be apparent:
• Index increments
• Timestamp is the string representation of the current time
• BPM is your rate you took earlier
• Difficulty is simply taken from the constant at the top of our program. You won’t use this field in this example but it’s useful to have if you’re doing further verification and you want to make sure the difficulty is consistent with the hash results (i.e. the hash result has N prefixed zeros so Difficulty should also equal N, or the chain is compromised)
The for loop is the critical piece in this function. Let’s walk through what’s happening here:
• You’ll take the hex representation of i and set Nonce equal to it. You need a way to append a changing value to your hash that you create from your calculateHash function so if you don’t get the leading number of zeros you want, you can try again with a new value. This changing value you add to your concatenated string in calculateHash is called a “Nonce”
• In your loop, you calculate the hash with i and Nonce starting at 0 and check if the result starts with the number of zeros as defined by your constant difficulty. If it doesn’t, you invoke the next iteration of the loop with an incremented Nonce and try again.
• You added a 1 second sleeper to simulate taking some time to solve the Proof of Work
• You keep looping until you get the number of leading zeros you want, which means you’ve successfully completed your Proof of Work. Then (and only then) you allow yyour Block to get added to blockchain via your handleWriteBlock handler
You’re done writing all your functions so let’s complete your main function now:
func main() {
err := godotenv.Load()
if err != nil {
log.Fatal(err)
}
go func() {
t := time.Now()
genesisBlock := Block{}
genesisBlock = Block{0, t.String(), 0, calculateHash(genesisBlock), “”, difficulty, “”}
spew.Dump(genesisBlock)
mutex.Lock()
Blockchain = append(Blockchain, genesisBlock)
mutex.Unlock()
}()
log.Fatal(run())
}
With godotenv.Load() you load up your environment variable, which is just the :8080 port you’ll access from your browser.
A go routine creates your genesis block since you need to supply your blockchain with a starting point.
You fire up your web server with your run() function you created earlier.
You’re done! Time to test it!
Start up your program with go run main.go
Then visit http://localhost:8080 in your browser:
Your genesis block has been created for you. Now open Postman and let’s send a POST request to the same route with your BPM (rate) we took earlier in a JSON body.
After you send the request watch what’s happening in your terminal. You’ll see your machine chugging away creating new hashes with incrementing Nonce values until it finally finds a result that has the required number of leading zeros!
When the Proof of Work is solved you get a helpful message saying work done! and you can check the hash to see that it indeed starts with the number of zeros you’ve set in difficulty. This means that in theory, the new block you tried to add with BPM = 60 should now have been added to your blockchain.
Refresh your browser and check:
SUCCESS! Your second block has been added to your genesis block. This means you successfully sent your block in a POST request, which triggered the mining process and ONLY when the Proof of Work was solved it got added to your blockchain.
Next steps
Nice work. Proof of Work is the foundation to Bitcoin, Ethereum and many of the biggest blockchain platforms around. What we just went through is not trivial; while we used a low difficulty for demonstration purposes, increasing the difficulty to a high number is exactly how production-ready Proof of Work blockchains work.
So what the heck is Proof of Stake?
Instead of nodes competing with each other to solve hashes, in Proof of Stake, blocks are forged (there is no mining so we don’t use that word in Proof of Stake) based on the amount of tokens each node is willing to put up as collateral. These nodes are called validators. The tokens are specific to the blockchain. So in Ethereum, each node (validator) would put up Ether as collateral.
The more tokens each validator is willing to put up as collateral, the greater the chance they have to forge the next block and receive a reward. You can think of this as deposit interest. The more money you put in your savings account at your bank, the greater the monthly interest payment you receive.
Similarly, your probability of forging the next block increases the more tokens you put up as collateral. You are “staking” your tokens, which is why this consensus mechanism is called Proof of Stake.
What are the downsides to Proof of Stake?
You can probably already guess that a validator who has an excessively large amount of tokens staked will enjoy a disproportionately high probability of forging new blocks. However, this isn’t really different from what we see in Proof of Work. Bitcoin mining farms are getting so powerful that regular people haven’t been able to mine on their own laptops in years. Thus, many people argue that Proof of Stake is actually more democratized since anyone can at least participate on their own laptops without setting up a giant mining rig. They don’t need expensive hardware, just enough tokens to stake.
There are other downsides to Proof of Stake, both from a technological and economic standpoint. In reality, Proof of Stake and Proof of Work both have their strengths and projects like Ethereum’s Casper blends features from both.
As always, the best way to understand how Proof of Stake works is to write your own code!
Let’s code a Proof of Stake blockchain.
• Full peer-to-peer implementation. Networking is simulated and the central blockchain state is held by a single Go TCP server.
• Wallet and balance tracking. I have not implemented a wallet in this code. Nodes are spun up in the network and the token amount is inputted in stdin. So you can type in any amount you want. A full implementation would associate each node with a hash address and keep track of token balances in each.
Architecture
• You’ll have a Go-based TCP server to which other nodes (validators) can connect.
• The latest blockchain state will get broadcast to each node periodically.
• Each node will propose new blocks.
• Based on the number of tokens staked by each node, one of the nodes will be randomly (weighted by the number of tokens staked) chosen as the winner, and their block will get added to the blockchain.
Before starting to write the code, you need to set an environment variable so your TCP server knows which port to use. Let’s create a .env file in your working directory with one line in it:
ADDR=9000
Your Go program will read from this file and know to expose port 9000 so your nodes can connect to it.
Now let’s create a main.go file in your working directory and start coding!
As usual, let’s make our package declaration and write up the imports you’ll need.
package main
import (
“bufio”
“crypto/sha256”
“encoding/hex”
“encoding/json”
“fmt”
“io”
“log”
“math/rand”
“net”
“os”
“strconv”
“sync”
“time”
“github.com/davecgh/go-spew/spew”
“github.com/joho/godotenv”
)
• spew is a convenient package that pretty prints our blockchain to the terminal
• godotenv allows you to read from your .env file you created earlier
Quick rate check
At this stage you want to add your rate.
Global variables
Now let’s declare all your global variables you’ll need.
// Block represents each ‘item’ in the blockchain
type Block struct {
Index int
Timestamp string
BPM int
Hash string
PrevHash string
Validator string
}
// Blockchain is a series of validated Blocks
var Blockchain []Block
var tempBlocks []Block
// candidateBlocks handles incoming blocks for validation
var candidateBlocks = make(chan Block)
// announcements broadcasts winning validator to all nodes
var announcements = make(chan string)
var mutex = &sync.Mutex{}
// validators keeps track of open validators and balances
var validators = make(map[string]int)
• Block is the content of each block
• Blockchain is your official blockchain, that is simply a series of validated blocks. The PrevHash in each block is compared to the Hash of the previous block to make sure your chain is robust. tempBlocks is simply a holding tank of blocks before one of them is picked as the winner to be added to Blockchain
• candidateBlocks is a channel of blocks; each node that proposes a new block sends it to this channel
• announcements is a channel where your main Go TCP server broadcasts the latest blockchain to all the nodes
• mutex is a standard variable that allows you to control reads/writes and prevent data races
• validators is a map of nodes and the amount of tokens they’ve staked
Basic blockchain functions
Before proceeding with your Proof of Stake algorithms, let’s write up your standard blockchain functions.
// SHA256 hasing
// calculateHash is a simple SHA256 hashing function
func calculateHash(s string) string {
h := sha256.New()
h.Write([]byte(s))
hashed := h.Sum(nil)
return hex.EncodeToString(hashed)
}
//calculateBlockHash returns the hash of all block information
func calculateBlockHash(block Block) string {
record := string(block.Index) + block.Timestamp + string(block.BPM) + block.PrevHash
return calculateHash(record)
}
You start with your hashing functions. calculateHash takes in a string and returns its SHA256 hash representation. calculateBlockHash hashes the contents of a block by concatenating all its fields.
// generateBlock creates a new block using previous block’s hash
func generateBlock(oldBlock Block, BPM int, address string) (Block, error) {
var newBlock Block
t := time.Now()
newBlock.Index = oldBlock.Index + 1
newBlock.Timestamp = t.String()
newBlock.BPM = BPM
newBlock.PrevHash = oldBlock.Hash
newBlock.Hash = calculateBlockHash(newBlock)
newBlock.Validator = address
return newBlock, nil
}
generateBlock is how a new block is created. The important fields you include in each new block are its hash signature (calculated by calculateBlockHash previously) and the hash of the previous block PrevHash(so you can keep the integrity of the chain). You also add a Validator field so you know the winning node that forged the block.
// isBlockValid makes sure block is valid by checking index
// and comparing the hash of the previous block
func isBlockValid(newBlock, oldBlock Block) bool {
if oldBlock.Index+1 != newBlock.Index {
return false
}
if oldBlock.Hash != newBlock.PrevHash {
return false
}
if calculateBlockHash(newBlock) != newBlock.Hash {
return false
}
return true
}
isBlockValid performs the Hash and PrevHash check to make sure your chain has not been corrupted.
Nodes (Validators)
When a validator connects to your TCP server, you need to provide it some functions that achieve a few things:
• Allow it to enter a token balance (remember you won’t perform any balance checks since there is no wallet logic)
• Receive a broadcast of the latest blockchain
• Receive a broadcast of which validator in the network won the latest block
• Add itself to the overall list of validators
• Enter block data BPM — remember, this is each validator’s rate
• Propose a new block
You’ll write all this up in a handleConn function. Here it is.
func handleConn(conn net.Conn) {
defer conn.Close()
go func() {
for {
msg := <-announcements
io.WriteString(conn, msg)
}
}()
// validator address
var address string
// allow user to allocate number of tokens to stake
// the greater the number of tokens, the greater chance to forging a new block
io.WriteString(conn, “Enter token balance:”)
scanBalance := bufio.NewScanner(conn)
for scanBalance.Scan() {
balance, err := strconv.Atoi(scanBalance.Text())
if err != nil {
log.Printf(“%v not a number: %v”, scanBalance.Text(), err)
return
}
t := time.Now()
address = calculateHash(t.String())
validators[address] = balance
fmt.Println(validators)
break
}
io.WriteString(conn, “\nEnter a new BPM:”)
scanBPM := bufio.NewScanner(conn)
go func() {
for {
// take in BPM from stdin and add it to blockchain after conducting necessary validation
for scanBPM.Scan() {
bpm, err := strconv.Atoi(scanBPM.Text())
// if malicious party tries to mutate the chain with a bad input, delete them as a validator and they lose their staked tokens
if err != nil {
log.Printf(“%v not a number: %v”, scanBPM.Text(), err)
delete(validators, address)
conn.Close()
}
mutex.Lock()
oldLastIndex := Blockchain[len(Blockchain)-1]
mutex.Unlock()
// create newBlock for consideration to be forged
newBlock, err := generateBlock(oldLastIndex, bpm, address)
if err != nil {
log.Println(err)
continue
}
if isBlockValid(newBlock, oldLastIndex) {
candidateBlocks <- newBlock
}
io.WriteString(conn, “\nEnter a new BPM:”)
}
}
}()
// simulate receiving broadcast
for {
time.Sleep(time.Minute)
mutex.Lock()
output, err := json.Marshal(Blockchain)
mutex.Unlock()
if err != nil {
log.Fatal(err)
}
io.WriteString(conn, string(output)+”\n”)
}
}
The first Go routine receives and prints out any announcements that come from the TCP server. These announcements will be who the winning validator is when one is chosen.
The section starting with io.WriteString(conn, “Enter token balance:”) allows the validator to enter the number of tokens he wants to stake. Then the validator is assigned a SHA256 address and that address is added to your global validators map you declared earlier, along with our new validator’s number of tokens staked.
You then enter BPM which is the validator’s rate and create a separate Go routine to process your block logic. The following line is important:
delete(validators, address)
If the validator tries to propose a polluted block, in your case, a BPM that is not an integer, that throws an error and you immediately delete the validator from your list of validators. They are no longer eligible to forge new blocks and they lose their balance.
This potential to lose your token balance is a major reason why Proof of Stake is generally secure. If you try to alter the blockchain for your benefit and you get caught, you lose your entire staked token balance. It’s a major deterrent for bad actors.
You then create a new block with the generateBlock function from before and you send it to the candidateBlocks channel for further processing. Sending stuff into a channel uses this syntax:
candidateBlocks <- newBlock
The last for loop periodically prints the latest blockchain so each validator knows the latest state.
Picking a Winner
This is the meat of Proof of Stake logic. You need to write up how a winning validator is chosen; the higher the number of tokens they stake, the higher their probability should be to be chosen as the winner who gets to forge their block.
For simplicity in your code, you will only make validators who propose new blocks eligible to be chosen as the winner. In traditional Proof of Stake, a validator can be chosen as the winner even if they don’t propose a new block. Remember, Proof of Stake isn’t a definition, it’s a concept; there are lots of different implementations of Proof of Stake and like Proof of Work, each implementation has its own nuances.
Here is our pickWinner function. I’ll walk you through it:
// pickWinner creates a lottery pool of validators and chooses the validator who gets to forge a block to the blockchain
// by random selecting from the pool, weighted by amount of tokens staked
func pickWinner() {
time.Sleep(30 * time.Second)
mutex.Lock()
temp := tempBlocks
mutex.Unlock()
lotteryPool := []string{}
if len(temp) > 0 {
// slightly modified traditional proof of stake algorithm
// from all validators who submitted a block, weight them by the number of staked tokens
// in traditional proof of stake, validators can participate without submitting a block to be forged
OUTER:
for _, block := range temp {
// if already in lottery pool, skip
for _, node := range lotteryPool {
if block.Validator == node {
continue OUTER
}
}
// lock list of validators to prevent data race
mutex.Lock()
setValidators := validators
mutex.Unlock()
k, ok := setValidators[block.Validator]
if ok {
for i := 0; i < k; i++ {
lotteryPool = append(lotteryPool, block.Validator)
}
}
}
// randomly pick winner from lottery pool
s := rand.NewSource(time.Now().Unix())
r := rand.New(s)
lotteryWinner := lotteryPool[r.Intn(len(lotteryPool))]
// add block of winner to blockchain and let all the other nodes know
for _, block := range temp {
if block.Validator == lotteryWinner {
mutex.Lock()
Blockchain = append(Blockchain, block)
mutex.Unlock()
for _ = range validators {
announcements <- “\nwinning validator: ” + lotteryWinner + “\n” } break } } } mutex.Lock() tempBlocks = []Block{} mutex.Unlock() } You pick a winner every 30 seconds to give time for each validator to propose a new block. Then you need to create a lotteryPool that holds addresses of validators who could be chosen as your winner. Then you check to see there actually are some blocks proposed in your temporary holding tank of proposed blocks with if len(temp) > 0 before proceeding with your logic.
In the OUTER for loop, you check to make sure you haven’t already come across the same validator in your temp slice. If you do, skip over the block and look for the next unique validator.
In the section starting with k, ok := setValidators[block.Validator] you make sure the validator you get from your block data in temp is actually an eligible validator that sits in your validators map. If they exist, then you add them to your lotteryPool .
How do you assign proper weights to them based on the number of tokens they staked?
• You fill your lotteryPool with copies of the validator’s address. They get a copy for each token they’ve staked. So a validator who put in 100 tokens will get 100 entries in the lotteryPool. A validator who only put in 1 token will only get 1 entry.
You randomly pick the winner from your lotteryPool and assign their address to lotteryWinner.
You then add their block to your blockchain and announce the winner to the rest of the nodes who won the lottery with this syntax, which sends the message into the announcements channel:
announcements <- “\nwinning validator: “ + lotteryWinner + “\n”
You clear out your tempBlocks holding tank so it can be filled again with the next set of proposed blocks.
That is the core of a Proof of Stake consensus algorithm! That wasn’t so bad was it?
Almost there!
Let’s wire up your main function now. Here it is:
func main() {
err := godotenv.Load()
if err != nil {
log.Fatal(err)
}
// create genesis block
t := time.Now()
genesisBlock := Block{}
genesisBlock = Block{0, t.String(), 0, calculateBlockHash(genesisBlock), “”, “”}
spew.Dump(genesisBlock)
Blockchain = append(Blockchain, genesisBlock)
// start TCP and serve TCP server
server, err := net.Listen(“tcp”, “:”+os.Getenv(“ADDR”))
if err != nil {
log.Fatal(err)
}
defer server.Close()
go func() {
for candidate := range candidateBlocks {
mutex.Lock()
tempBlocks = append(tempBlocks, candidate)
mutex.Unlock()
}
}()
go func() {
for {
pickWinner()
}
}()
for {
conn, err := server.Accept()
if err != nil {
log.Fatal(err)
}
go handleConn(conn)
}
}
You start by slurping in the contents of your .env file which is just the port number you’ll use for your TCP server. You then create a genesisBlock that new blocks will get added to, to form your blockchain.
You start your TCP server and expose the port in your .env file to which new validators can connect.
You start up a Go routine to take blocks out of your candidateBlocks channel and fill your tempBlocks holding tank with them, for further processing by your pickWinner function you just wrote. You then start up another Go routine for the pickWinner function.
The last for loop accepts connections from new validators.
You’re done!
The fun stuff
Let’s try this out! Open a terminal window and start up your Go program and TCP server with go run main.go . As expected, you get your genesisBlock printed on your console.
Now let’s fire up a validator. Open a new terminal window and connect to your TCP server with nc localhost 9000
You’re then prompted to add a token balance to stake. Enter the number of tokens you want that validator to stake. Then input a pulse rate for that validator.
Since you can have many validators, let’s do the same thing with another terminal window.
Watch your first terminal as you’re adding new terminals. You see validators get assigned addresses and you get a list of validators each time a new one is added.
Wait a little while and check out your new terminals. What’s happening is that your program is spending some time picking a winner. And then boom! A winner is chosen.
In your case, the first validator was chosen (you can verify the validator’s address by comparing it to the list of validators printed in your main terminal).
Wait a little while again and boom! You see your new blockchain broadcast to all your terminals, with your winning validator’s block containing his BPM in the newest block! Cool, right?
Next steps
You should be proud of getting through this example. Most blockchain enthusiasts and many programmers have heard of Proof of Stake but couldn’t even begin to explain what it is. You’ve gone even further and actually built a Proof of Stake blockchain from scratch!
Because this is a simple example, there is more you can do to make this a production ready blockchain. Things to explore next are:
• Add time chunks where validators have a chance to propose new blocks. Your version of the code lets validators propose new blocks anytime so some blocks may periodically get cut off from consideration
• Add full peer-to-peer capability. This would basically mean each validator would run its own TCP server as well as connecting to others. You would need to add in logic so each node can find each other.
This makes storing data or large files on the blockchain a non-starter. If the blockchain can barely sustain small strings of text that simply record a balance transfer between two parties, how on earth are we ever going to store large files or images on the blockchain? Are we just going to have to be OK with limiting the utility of the blockchain to things that can only be captured in tiny text strings?
Enter IPFS
The most promising solution that’s available today is IPFS, or Interplanetary File System, created by folks at Protocol Labs. It’s a peer-to-peer protocol where each node stores a collection of hashed files. A client who wants to retrieve any of those files enjoys access to a nice abstraction layer where it simply needs to call the hash of the file it wants. IPFS then combs through the nodes and supplies the client with the file.
You can think of it as being similar to BitTorrent. It’s a decentralized way of storing and referring to files but gives you more control and refers to files by hashes, allowing for much richer programmatic interactions.
Here are some simple diagrams so you can see the workflow of IPFS.
1. David wants to upload a PDF file to IPFS
2. He puts his PDF file in his working directory
3. He tells IPFS he wants to add this file, which generates a hash of the file (you can tell it’s IPFS because the hash always starts with Qm…)
4. His file is available on the IPFS network
Now suppose David wants to share this file with his colleague Sabrina through IPFS. He simply tells Sabrina the hash from Step 3 above. Then steps 1–4 above just work in reverse for Sabrina. All Sabrina needs to do is call the hash from IPFS and she gets a copy of the PDF file. Sweet.
Security Hole
There is an obvious security hole here. As long as anyone has the hash of the PDF file, they can retrieve it from IPFS. So sensitive files are not well suited for IPFS in their native states. Unless we do something to these files, sharing sensitive files like private records or images is a poor fit for IPFS.
Enter Asymmetric Encryption
Luckily, we have tools at our disposable that pair very nicely with IPFS to secure files before uploading them to IPFS. Asymmetric encryption allows us to encrypt a file with the public key of the intended recipient so that only they can decrypt it when they retrieve it with IPFS. A malicious party who retrieves the file from IPFS can’t do anything with it since they can’t decrypt it. For this example we’ll be using GPG for asymmetric encryption.
Let’s edit our workflow diagram a bit so we include encryption and decryption:
1. David wants to upload a PDF file to IPFS but only give Sabrina access
2. He puts his PDF file in his working directory and encrypts it with Sabrina’s public key
3. He tells IPFS he wants to add this encrypted file, which generates a hash of the encrypted file
4. His encrypted file is available on the IPFS network
5. Sabrina can retrieve it and decrypt the file since she owns the associated private key of the public key that was used to encrypt the file
6. A malicious party cannot decrypt the file because they lack Sabrina’s private key
The Blockchain
So where does the blockchain fit into this?
Of particular importance is this diagram:
Pay attention to the BPM part. This kind of simple text recording is all the blockchain can really handle today. This is why cryptocurrencies are a good fit for the blockchain. All you need to record is the sender, recipient and amount of Bitcoin (or Ether, etc.) being transferred. Because all these hashes need to be calculated and verified to preserve integrity of the chain, the blockchain is horrible, absolutely horrible at storing files or large amounts of data in a block.
This is why IPFS is so powerful when coupled with the blockchain. Instead of BPM above, we simply store the hash of the IPFS file. We keep the simplicity of data that’s required on the blockchain but we get to enjoy the file storage and decentralized peer-to-peer properties of IPFS. It’s the best of both worlds. Since we also added security with asymmetric encryption (GPG), we have a very elegant way of “storing”, encrypting, and sharing large data and files on the blockchain.
A real world application would be storing referents to our person or private records in each block. When you get a new carrier result, we simply create a new block that refers to an encrypted image or PDF of your carrier result that sits in IPFS.
Enough talk. I show you how to do this.
In this example you will cover the following:
• Set up GPG
• Set up IPFS
• Encrypt a file with someone else’s public key
• Upload the encrypted file to IPFS
• Download the file from another computer (or Virtual Machine) and make sure only the privileged party can decrypt and view it
Things you’ll need
• A second computer or a Virtual Machine instance. The second computer simulates a person with whom you want to securely share your files.
• A test file. Put this file in your working directory.
That’s it! Let’s get started.
Setup
GPG
Download GPG on both your main and secondary computer/s.
On Mac, the easiest way is to open your terminal and brew install gnupg assuming Homebrew is installed.
Generate a key on each of your computers after GPG installation. Use the following steps:
gpg –gen-key and follow the prompts and pick the default options. Make sure to securely remember or store the password you choose for your username and email.
You’ll get to a stage where gpg asks you to do some random things to generate entropy. I just typed a bunch of random characters until the process was finished.
After the key has been generated on the second computer, you need to add that key to the keyring of the first computer, so you can encrypt files that only the second computer can decrypt.
Export your public key on your second computer into an armored blob using the email address you chose when creating the key
gpg –export –armor -email > pubkey.asc
Move the pubkey.asc file you just created to your first computer. Make sure to do this securely. A USB stick is better than sending it over email.
Once the pubkey.asc file is on your first computer and your working directory, import it into your keyring like this
gpg –import pubkey.asc
You can check to see it was imported correctly with gpg –list-keys. I gave my second computer’s a random name and it shows up correctly:
Great! You’re done with GPG setup. Let’s move onto IPFS.
IPFS
Initialize IPFS with ipfs init on both computers and start your daemon with ipfs daemon on both computers:
Nice! You’ve set everything up. Let’s get to encrypting and uploading your PDF file to IPFS.
Encryption
Remember the sample carrier result you downloaded earlier? Make sure to move that to your working directory on your first computer.
Let’s encrypt that file (I renamed it goshaytrading.pdf since the carrier result was produced by Go’Shay Trading) using the public key of the 2nd computer.
gpg –encrypt –recipient “Corth” goshaytrading.pdf
If you check your directory now with ls you’ll see a new encrypted file named goshaytrading.pdf.gpg
Only your second computer can decrypt and see this file. Try it! Email it to another friend and try as they might, they won’t be able to open it! Even if they rename it back to goshaytrading.pdf
You’ve got your encrypted file now. Upload it to IPFS.
Uploading to IPFS
To upload to IPFS, all you need to do on your first computer is
ipfs add goshaytrading.pdf.gpg
The Qm… string is the hash of the file. You can send this to your friend or anyone to whom you wish to give access so they can download it from IPFS.
Let’s just double check to make sure your file is available on IPFS with ipfs pin ls
You can see the hash of your file is indeed present and now available on IPFS.
Downloading from IPFS
Let’s now switch to your second computer. Remember, you are simulating a second person. To make this more realistic, ask a friend to work on the second computer.
In my case, instead of a second computer I’m using an Ubuntu VM with Vagrant. This is not a must.
On your second computer, download the posted encrypted file from your first computer from IPFS using the same hash:
ipfs get QmYqSCWuzG8Cyo4MFQzqKcC14ct4ybAWyrAc9qzdJaFYTL
This is what it should look like when successfully downloaded:
Decryption
Since you’re on your second computer, and this encrypted file was encrypted with the second computer’s public key, in theory you should be able to decrypt and view this file without any issues.
Let’s give it a try.
Decrypt the downloaded file and rename it to goshaytrading.pdf
gpg –decrypt QmYqSCWuzG8Cyo4MFQzqKcC14ct4ybAWyrAc9qzdJaFYTL > goshaytrading.pdf
Moment of truth:
Open this file and if all went well you should be able to see it on your second computer.
open goshaytrading.pdf
BOOM! You successfully downloaded, decrypted and opened your file which was stored fully encrypted on IPFS, protected from anyone who shouldn’t have access.
Recap and Next Steps
Give yourself a pat on the back. What you just accomplished is incredibly powerful and addresses some key issues found in blockchain technology today.
Where you take what you learned here is completely up to you. There are many places to branch off from this. Consider deploying these examples to live servers to act as your own IPFS nodes to store important files. The drawback to IPFS is that if your files aren’t very popular, when you stop your node, your file is gone from the IPFS network. You can prevent this by spinning up cloud servers to act as their own IPFS nodes, so you can host them yourself until more nodes become interested in your files and start storing them.
Let’s get started!
Background
What is Peer-to-Peer (P2P)?
In true Peer-to-Peer architecture, you don’t need a central server to maintain the state of a blockchain. For example, when you send your friend some Bitcoin, the “state” of the Bitcoin blockchain should be updated so your friend’s balance increases and your balance decreases.
There isn’t a central authority like a bank that maintains the state. Instead, all the nodes in the Bitcoin network that wish to maintain a copy of the Bitcoin blockchain update their copies of the blockchain to include your transaction. That way, as long as 51% of the nodes in the network “agree” on the state of the blockchain, it maintains its fidelity.
Let’s code!
Coding a P2P network is no joke, nor for newbies. It has a ton of edge cases and requires a lot of engineering to make it scalable and reliable. Like any good engineer, we will first see what tools are available as a starting point.
Luckily, there is an excellent P2P library written in Go called go-libp2p. Coincidentally, it is made by the same folks who created IPFS.
Warning
As far as I can tell, this go-libp2p library has 2 drawbacks:
1. Setup is a pain. It uses gx as a package manager that we don’t find very convenient.
2. It appears to still be under constant development. When working with the code, you will encounter some minor data races. They have some glitches to iron out.
Don’t worry about #1. I’ll help you through it. #2 is a bigger issue but won’t affect our code. However, if you do notice data races, they are most likely coming from the underlying code of this library. So make sure to open an issue and let them know.
There are very few modern, open source P2P libraries available, particularly in Go. Overall, go-libp2p is quite good and it’s well suited to our objectives.
Setup
The best way to get your code environment set up is to clone this entire library and write your code within it. You can develop outside the environment they provide but it requires knowing how to work with gx. I’ll show you the easy way. Assuming you already have Go installed:
• go get -d github.com/libp2p/go-libp2p/…
• navigate to the cloned directory from above
• make
• make deps
This gets you all the packages and dependencies you need from this repo through the gx package manager. Again, you don’t like gx as it breaks a lot of Go convention (besides, why not just stick with go get?) but it’s worth the inconvenience to use this otherwise nice library.
You’re going to develop inside the examples subdirectory. So create a directory under examples called p2p with
• mkdir ./examples/p2p
Then navigate to your new p2p folder and create a main.go file. You will be writing all your code in this file.
Your directory tree should look like this:
Open up your main.go file and start writing your code.
Imports
Let’s make your package declaration and list your imports. Most of these imports are packages provided by the go-libp2p library. You’ll learn how to use each of them throughout this example.
package main
import (
“bufio”
“context”
“crypto/rand”
“crypto/sha256”
“encoding/hex”
“encoding/json”
“flag”
“fmt”
“io”
“log”
mrand “math/rand”
“os”
“strconv”
“strings”
“sync”
“time”
“github.com/davecgh/go-spew/spew”
golog “github.com/ipfs/go-log”
libp2p “github.com/libp2p/go-libp2p”
crypto “github.com/libp2p/go-libp2p-crypto”
host “github.com/libp2p/go-libp2p-host”
net “github.com/libp2p/go-libp2p-net”
peer “github.com/libp2p/go-libp2p-peer”
pstore “github.com/libp2p/go-libp2p-peerstore”
ma “github.com/multiformats/go-multiaddr”
gologging “github.com/whyrusleeping/go-logging”
)
The spew package is a simple convenience package to pretty print your blockchain. Make sure to:
• go get github.com/davecg/go-spew/spew
Blockchain stuff
Let’s declare your global variables.
// Block represents each ‘item’ in the blockchain
type Block struct {
Index int
Timestamp string
BPM int
Hash string
PrevHash string
}
// Blockchain is a series of validated Blocks
var Blockchain []Block
var mutex = &sync.Mutex{}
• Block is the transaction info you want. You are using BPM (your rate) as the key data point going in each block. Take your rate and remember that number.
• Blockchain is your “state”, or the latest blockchain, which is just a slice of Block
• You declare a mutex so youy can control for and prevent race conditions in your code
Write out the following blockchain-specific functions.
// make sure block is valid by checking index, and comparing the hash of the previous block
func isBlockValid(newBlock, oldBlock Block) bool {
if oldBlock.Index+1 != newBlock.Index {
return false
}
if oldBlock.Hash != newBlock.PrevHash {
return false
}
if calculateHash(newBlock) != newBlock.Hash {
return false
}
return true
}
// SHA256 hashing
func calculateHash(block Block) string {
record := strconv.Itoa(block.Index) + block.Timestamp + strconv.Itoa(block.BPM) + block.PrevHash
h := sha256.New()
h.Write([]byte(record))
hashed := h.Sum(nil)
return hex.EncodeToString(hashed)
}
// create a new block using previous block’s hash
func generateBlock(oldBlock Block, BPM int) Block {
var newBlock Block
t := time.Now()
newBlock.Index = oldBlock.Index + 1
newBlock.Timestamp = t.String()
newBlock.BPM = BPM
newBlock.PrevHash = oldBlock.Hash
newBlock.Hash = calculateHash(newBlock)
return newBlock
}
• isBlockValid checks to see that the chain of hashes in each block of the blockchain is consistent
• calculateHash uses sha256 to hash raw data
• generateBlock creates a new block to be added to the blockchain, with the necessary transaction info inside it
P2P Stuff
Host
The first thing you want to do is write the logic that allows a host to be created. When a node runs your Go program, it should act as a host to which other nodes (or peers) can connect. Here’s the code.
// makeBasicHost creates a LibP2P host with a random peer ID listening on the
// given multiaddress. It will use secio if secio is true.
func makeBasicHost(listenPort int, secio bool, randseed int64) (host.Host, error) {
// If the seed is zero, use real cryptographic randomness. Otherwise, use a
// deterministic randomness source to make generated keys stay the same
// across multiple runs
var r io.Reader
if randseed == 0 {
r = rand.Reader
} else {
r = mrand.New(mrand.NewSource(randseed))
}
// Generate a key pair for this host. We will use it
// to obtain a valid host ID.
priv, _, err := crypto.GenerateKeyPairWithReader(crypto.RSA, 2048, r)
if err != nil {
return nil, err
}
opts := []libp2p.Option{
libp2p.ListenAddrStrings(fmt.Sprintf(“/ip4/127.0.0.1/tcp/%d”, listenPort)),
libp2p.Identity(priv),
}
if !secio {
opts = append(opts, libp2p.NoEncryption())
}
basicHost, err := libp2p.New(context.Background(), opts…)
if err != nil {
return nil, err
}
// Build host multiaddress
hostAddr, _ := ma.NewMultiaddr(fmt.Sprintf(“/ipfs/%s”, basicHost.ID().Pretty()))
// Now we can build a full multiaddress to reach this host
// by encapsulating both addresses:
addr := basicHost.Addrs()[0]
fullAddr := addr.Encapsulate(hostAddr)
log.Printf(“I am %s\n”, fullAddr)
if secio {
log.Printf(“Now run \”go run main.go -l %d -d %s -secio\” on a different terminal\n”, listenPort+1, fullAddr)
} else {
log.Printf(“Now run \”go run main.go -l %d -d %s\” on a different terminal\n”, listenPort+1, fullAddr)
}
return basicHost, nil
}
Your makeBasicHost function takes 3 arguments and returns the host and an error (which is nul if no errors are encountered).
• listenPort is the port you specify in your command line flags that other peers connect to
• secio is a boolean that turns on and off secure data streams. It’s generally a good idea to use it. It stands for “secure input/output”
• randSeed is an optional command line flag that allows you to supply a seed to create a random address for your host. You won’t use this but it’s nice to have.
The function’s first if statement determines whether the seed was supplied and generates keys for your host accordingly. You then generate your public and private keys so your host stays secure. The opts section starts to construct the address to which other peers can connect.
The !secio part bypasses encryption but you’re going to use secio for security so this line doesn’t apply to you for now. It’s fine to have this option though.
You then create the host and finalize the addresses that other peers can connect to. The log.Printf part at the end is the helpful console message you print that tells a new node how to connect to the host you just created.
You then return the fully created host to the caller of the function. You now have your host!
Stream handler
You need to allow your host to deal with incoming data streams. When another node connects to your host and wants to propose a new blockchain to overwrite your own, you need logic to determine whether or not you should accept it.
When you add blocks to your blockchain, you want to broadcast it to your connected peer, so you need logic to do that as well.
Let’s create the skeleton of your handler.
func handleStream(s net.Stream) {
log.Println(“Got a new stream!”)
// Create a buffer stream for non blocking read and write.
rw := bufio.NewReadWriter(bufio.NewReader(s), bufio.NewWriter(s))
go readData(rw)
go writeData(rw)
// stream ‘s’ will stay open until you close it (or the other side closes it).
}
You create a new ReadWriter since you need both read and write and you spin up separate Go routines to deal with the read and write logic.
Read
Let’s create your readData function first.
func readData(rw *bufio.ReadWriter) {
for {
str, err := rw.ReadString(‘\n’)
if err != nil {
log.Fatal(err)
}
if str == “” {
return
}
if str != “\n” {
chain := make([]Block, 0)
if err := json.Unmarshal([]byte(str), &chain); err != nil {
log.Fatal(err)
}
mutex.Lock()
if len(chain) > len(Blockchain) {
Blockchain = chain
bytes, err := json.MarshalIndent(Blockchain, “”, ” “)
if err != nil {
log.Fatal(err)
}
// Green console color: \x1b[32m
// Reset console color: \x1b[0m
fmt.Printf(“\x1b[32m%s\x1b[0m> “, string(bytes))
}
mutex.Unlock()
}
}
}
Your function is an infinite loop since it needs to stay open to incoming blockchains. You parse your incoming blockchain from a peer which is just a string of a JSON blob with ReadString. If it’s not empty (!= “\n”) you Unmarshal the blob first.
Then you check to see if the length of the incoming chain is longer than the blockchain you’re storing youself. For this purposes, you are simply going by blockchain length to determine who wins out. If the incoming chain is longer than yours, you’ll accept it as the latest network state (or the latest, “true” blockchain).
You’ll Marshal it back into a JSON format so it’s easier to read then we print it to your console. The fmt.Printf command prints in a different color so it’s easy for you to know it’s a new chain.
Now you’ve accepted your peer’s blockchain, if youy add a new block to your blockchain, you need a way to let your connected peer know about it, so they can accept yours. You do that with your writeData function.
Write
func writeData(rw *bufio.ReadWriter) {
go func() {
for {
time.Sleep(5 * time.Second)
mutex.Lock()
bytes, err := json.Marshal(Blockchain)
if err != nil {
log.Println(err)
}
mutex.Unlock()
mutex.Lock()
rw.WriteString(fmt.Sprintf(“%s\n”, string(bytes)))
rw.Flush()
mutex.Unlock()
}
}()
stdReader := bufio.NewReader(os.Stdin)
for {
fmt.Print(“> “)
sendData, err := stdReader.ReadString(‘\n’)
if err != nil {
log.Fatal(err)
}
sendData = strings.Replace(sendData, “\n”, “”, -1)
bpm, err := strconv.Atoi(sendData)
if err != nil {
log.Fatal(err)
}
newBlock := generateBlock(Blockchain[len(Blockchain)-1], bpm)
if isBlockValid(newBlock, Blockchain[len(Blockchain)-1]) {
mutex.Lock()
Blockchain = append(Blockchain, newBlock)
mutex.Unlock()
}
bytes, err := json.Marshal(Blockchain)
if err != nil {
log.Println(err)
}
spew.Dump(Blockchain)
mutex.Lock()
rw.WriteString(fmt.Sprintf(“%s\n”, string(bytes)))
rw.Flush()
mutex.Unlock()
}
}
You start the function off with a Go routine that broadcasts the latest state of your blockchain every 5 seconds to your peers. They’ll receive it and throw it away if the length is shorter than theirs. They’ll accept it if it’s longer. Either way, all the peers are constantly getting their blockchains updated by the latest state of the network.
You now need a way to create a new block with the rate (BPM) you took earlier. You create a new reader with bufio.NewReader so it can read your stdin (console input). You want to be able to continually add new blocks so you put this in an infinite loop.
You do a little bit of string manipulation to make sure the BPM you type in is an integer and properly formatted to be added as a new block. You go through your standard blockchain functions. Then you Marshal it so it looks pretty, print it to your console to verify with spew.Dump. You then broadcast it to your connected peer with rw.WriteString.
Great! We’ve now finished your blockchain functions and most of your P2P functions. You’ve created your handler, and the read and write logic to deal with incoming and outgoing blockchains. Through these functions you’ve created a way for each peer to continually check the state of their blockchain against each other and collectively, they all get updated to the latest state (longest valid blockchain).
All that’s left now is wiring up your main function.
Main function
Here is your main function. Have a look through it first then I’ll go through it step by step.
func main() {
t := time.Now()
genesisBlock := Block{}
genesisBlock = Block{0, t.String(), 0, calculateHash(genesisBlock), “”}
Blockchain = append(Blockchain, genesisBlock)
// LibP2P code uses golog to log messages. They log with different
// string IDs (i.e. “swarm”). We can control the verbosity level for
// all loggers with:
golog.SetAllLoggers(gologging.INFO) // Change to DEBUG for extra info
// Parse options from the command line
listenF := flag.Int(“l”, 0, “wait for incoming connections”)
target := flag.String(“d”, “”, “target peer to dial”)
secio := flag.Bool(“secio”, false, “enable secio”)
seed := flag.Int64(“seed”, 0, “set random seed for id generation”)
flag.Parse()
if *listenF == 0 {
log.Fatal(“Please provide a port to bind on with -l”)
}
// Make a host that listens on the given multiaddress
ha, err := makeBasicHost(*listenF, *secio, *seed)
if err != nil {
log.Fatal(err)
}
if *target == “” {
log.Println(“listening for connections”)
// Set a stream handler on host A. /p2p/1.0.0 is
// a user-defined protocol name.
ha.SetStreamHandler(“/p2p/1.0.0”, handleStream)
select {} // hang forever
/**** This is where the listener code ends ****/
} else {
ha.SetStreamHandler(“/p2p/1.0.0”, handleStream)
// The following code extracts target’s peer ID from the
// given multiaddress
ipfsaddr, err := ma.NewMultiaddr(*target)
if err != nil {
log.Fatalln(err)
}
pid, err := ipfsaddr.ValueForProtocol(ma.P_IPFS)
if err != nil {
log.Fatalln(err)
}
peerid, err := peer.IDB58Decode(pid)
if err != nil {
log.Fatalln(err)
}
// Decapsulate the /ipfs/ part from the target
// /ip4//ipfs/ becomes /ip4/
targetPeerAddr, _ := ma.NewMultiaddr(
fmt.Sprintf(“/ipfs/%s”, peer.IDB58Encode(peerid)))
targetAddr := ipfsaddr.Decapsulate(targetPeerAddr)
// We have a peer ID and a targetAddr so we add it to the peerstore
// so LibP2P knows how to contact it
ha.Peerstore().AddAddr(peerid, targetAddr, pstore.PermanentAddrTTL)
log.Println(“opening stream”)
// make a new stream from host B to host A
// it should be handled on host A by the handler we set above because
// we use the same /p2p/1.0.0 protocol
s, err := ha.NewStream(context.Background(), peerid, “/p2p/1.0.0”)
if err != nil {
log.Fatalln(err)
}
// Create a buffered stream so that read and writes are non blocking.
rw := bufio.NewReadWriter(bufio.NewReader(s), bufio.NewWriter(s))
// Create a thread to read and write data.
go writeData(rw)
go readData(rw)
select {} // hang forever
}
}
You start off by creating a Genesis block, which is your seed block for your blockchain.
You use the go-libp2p library’s logger to handle logging with SetAllLoggers. This is optional.
You then set all your command line flags.
• secio I covered previously and allows for secure streams. You’ll make sure to always use this by setting the flag when running our program.
• target lets you specify the address of another host you want to connect to, which means you’re acting as a peer to a host if you use this flag.
• listenF opens the port you want to allow connections to, which means you’re acting as a host. You can be both a host (receive connections) and a peer (connect to other hosts). This is what makes this system truly P2P!
• seed is the optional randomness seeder used to construct your address that other peers can use to connect to you.
You then create a new host with the makeBasicHost function you created earlier. If you’re only acting as a host (i.e. you’re not connecting to other hosts) you specify that with if *target == “” , fire up your handler with the setStreamHandle function you made earlier and that’s the end of your listener code.
If you do want to connect to another host you move onto the else section. You set your handler again, since you’re acting as a host AND a connecting peer.
The next few lines deconstruct the string you supply to target so you can find the host you want to connect to. This is also called Decapsulation.
You end up with the peerID and target address targetAddr of the host you want to connect to and add that record into your “store” so you can keep track of who you’re connected to. You do that with ha.Peerstore().AddAddr
You then create your connection stream to the peer you want to connect to with ha.NewStream. You want to be able to receive and send them streams of data (our blockchain) so just like you did in your handler, you create a ReadWriter and spin up separate Go routines for readData and writeData. You finish off by blocking with an empty select statement so your program doesn’t just finish and quit.
Sweet!
You’re done! I know that was a bit of a handful but considering how complicated P2P engineering is you should be proud that you made it to the end! That wasn’t too bad was it?
Test Drive
Now you’re going to be using 3 separate terminals to act as individual peers.
Before starting the app, do yourself a favor and go to the root directory of go-libp2p and run make deps again. This makes sure your dependencies are in order. Again, this library uses the annoying gx package manager and we have to do a couple things to make it play nice.
Go back to your working directory.
On your first terminal, go run main.go -l 10000 -secio
Terminal 1
Follow the instructions that say “Now run…”. Open up a 2nd terminal, go to the same directory and
go run main.go -l 10001 -d -secio
Terminal 2
You’ll see the first terminal detected the new connection.
Terminal 1
Now following the instructions in the 2nd terminal, open up a 3rd terminal, go to the same working directory and
go run main.go -l 10002 -d -secio
Terminal 3
Check out the 2nd terminal, which detected the connection from the 3rd terminal.
Terminal 2
Now let’s start inputting our BPM data. Type in “70” in our 1st terminal, give it a few seconds and watch what happens in each terminal.
Terminal 1
Terminal 2
Terminal 3
What just happened here? It’s really cool so let’s think it through.
• Terminal 1 added a new block to its blockchain
• It then broadcast it to Terminal 2
• Terminal 2 compared it against its own blockchain, which only contained its genesis block. It saw Terminal 1 had a longer chain so it replaced its own chain with Terminal 1’s chain. Then it broadcast the new chain to Terminal 3.
• Terminal 3 compared the new chain against its own and replaced it.
All 3 terminals updated their blockchains to the latest state with no central authority! This is the power of Peer-to-Peer.
Let’s test it again but this time allow Terminal 2 to add a block. Type in “80” into Terminal 2.
Terminal 2
Terminal 1
Terminal 3
Awesome! As expected, this time Terminal 2 added a new block and broadcast it to the rest of the network. Each of the peers ran its own internal checks and updated their blockchains to the latest blockchain of the network!
Next Steps
Take a breathe and enjoy your work. You just coded up a simple but fully functioning P2P blockchain. This is no joke. P2P programming is complicated, which is why you don’t see many simple tutorials out there on how to create your own P2P network.
Now challenge yourself and try to tackle one or more of these improvements:
• As mentioned, the go-libp2p library is not perfect. You were careful to ensure your own code has minimal (if any) data races but while you were testing this library you noticed they had some data races in their code. It didn’t affect the results you were expecting to see but be extra cautious if you’re using this external library in production. Let them know about bugs you find as their library is a work in progress.
• Try incorporating consensus mechanisms to this example. You just did a simple check to see which blockchain was the longest, but you could incorporate either your Proof of Work or Proof of Stake scripts to have a more robust check.
• Add persistence to the code. Right now, for simplicity, you shut down all the nodes if you kill one of them. You can leave all other nodes running even if one closes. If one node dies, it should be able to relay the info in its Peer Store to its connected nodes so they can connect to each other.
• This code hasn’t been tested with hundreds of nodes. Try writing a shell script to scale many nodes and see how performance is affected. If you notice any bugs, make sure to file an issue or submit a pull request in your Github repo.
• Look into node discovery. How do new nodes find existing nodes? I know it, but the question is for you.
Section 7: Live to Trade
To overcome these price manipulators you need to understand the basic objectives of their activity. Overall, institutions are traders and their objective is to make money (this should be everyone’s objective too!). This includes strategies to trade against retails traders. The major difference between them and other traders is that they have the ability, through access to massive volumes, to move price at their will. So to make money, they aim to buy at a lower price and then sell at a higher price. They achieve this by:
1. Inducing traders to take positions.
This is achieved by using a range of price movements to trick traders into taking a position in a given direction and then reversing
it again. This means that the maker can sell a specific cryptocurrency at a certain price and then buy it back at a lower price when the retail trader feels too much pain from the cryptocurrency value moving backward and wanting to sell it back again (i.e. hitting all stop loss)
2. Create panic and fear to induce traders to become emotional and think irrationally.
This often involves:
– quick moves
– spike candles
– news releases
– Non-logic price behaviour.
3. Hit the Stops and Clear the Board.
This forces traders into margin troubles with their brokers and to be kicked out of the game.
WHICH TOOLS DO THE INSTITUTIONS HAVE?
Even though they have a number of tools at their disposal, they do have some restrictions imposed on them from outside authorities. The tool which mostly they utilize is to be able to buy or sell currency in different volumes at different price levels. By doing this strategically, they can:
1. Induce traders to take positions by providing evidence that price is moving or going to move in a certain direction.
2. Appeal to the emotional side of traders by changing the character and speed of price changes (long volume candles).
3. Once the trap has been set, and the bait taken, cause the price to move in such a way as to cause price to move against the traders, allowing the banks to buy currency back from or sell currency back to the retail traders so that they are square again.
4. This means that the trader has entered the market by buying currency from the bank at a given price and exited the market by selling back to the bank at a lower price. Conversely, the bank has sold to the trader at the higher given price and bought back from the trader at the lower price.
While these price movements are used to trap traders into unfavourable positions, they are not used 24 hours a day, but will be used more at certain times.
In addition to move prices institutions can take advantage of the so-called “sentiment” which develops as a result of news releases of various types. Sometimes immediate price movements are designed and used to cover-up makers’ price movement. The rumour mill also has a role to play in generating a public expectation of price movement. It is not uncommon to see the general news being particularly pessimistic for example about a particular currency only to see the currency rise against it but usually after people have been trapped in line with the sentiment bias.
WHICH TOOLS DO THE DEALERS AND BROKERS HAVE?
Brokers and dealers have mechanisms available to them for manipulating price to enable the process of taking money from traders, who are also their clients!
Usually, trader’s transactions are dealt with in house and never make it to the interbank market so it is very easy for them to manipulate price to their own advantage. They have a number of additional tools at their disposal and include:
– Requoting
– Trigger all stops in a given price range (which is part of the dealers functions on their own platforms)
– Vary the spread (which is why scalping methods often fail) at times when it is an advantage to them to do so.
– Throw a price spike to take stops out, bear in mind that they know where the stops are (all retail traders play on dealers platforms).
– Target traders who are in margin trouble and move price against their positions to finish them off. Again bear in mind that they
know who is in trouble because it is part of their backend platform.
ANATOMY OF AN M AND W FORMATION
The M or W pattern formation is a frequently identified pattern and is a particularly good reversal pattern. The following diagram shows the reasons for the movement in terms of the institutions use of the pattern. It is also worth noting that a RRT pattern is really an M or W pattern that has occurred more quickly and thus has the same effect.
The time gap between the 2 peaks of the M or W will usually last for somewhere between 30 and 90 min (though occasionally longer). The fastest occurs when the pattern is defined by a railroad track (in other words 15 min up followed by 15 min down on M15 chart). Longer periods are also common and used to gradually accumulate more positions of traders who are enticed into taking trades in the direction of the technical trend.
TRAPPING VOLUME
At a peak formation low or a peak formation high, several spikes may appear which are all apparently contained by a trend-line.
But what is really happening here?
The institutions are trapping volume and it is important to notice that each subsequent spike it is not lower (or higher) than the previous so that any new trades taken in the direction of the spike do not have an opportunity to become profitable. They become trapped.
So in the example below, the peak low is identified and followed by 2 further downward spikes. The important feature to notice is that each of the spikes is higher than the previous which prevents short position holders from taking any profit whilst potentially encouraging new shorts in this region.
THE WEDGE REDEFINED AS A VOLUME TRAPPING MECHANISM
In a similar way to the mechanism of trapping volume described previously, a wedge or pennant pattern works the same way except that it is trapping volume in both directions.
In the example below, you can observe that on the lower boundary of the wedge, the peaks become slightly higher each time it comes down to the line. This has the effect of ensuring that none of the trades that are taken short in these regions can turn a profit. Similarly, on the upper boundary of the wedge, the same thing is happening with each of the peaks becoming progressively lower and trapping the higher level longs and pulling them down.
There is no way of predicting which direction the price will ultimately breakout. This will be determined by the net volumes that occur. In other words, if there is a greater build-up of short positions over the long positions, then the wedge will break up (only dealers and makers know the true volume and open positions listed).
MAINTAINING THE VALIDITY OF HIGHS AND LOWS
As long as the price stays above the low after your entry or below the high after your entry, then the entry and therefore the trade remain valid.
Some caution needs to be applied when interpreting price movement that exceeds the high or low but closes back inside the noted high or low price. In these situations, the market-makers have spiked the price beyond the high or low to both trigger stops as well as to further induce traders to enter in the wrong direction.
FALSE SUPPORT AND RESISTANCE LEVELS
Consolidation zones are created during the cycle to create support and resistance levels which become visible on the charts and are used by retail traders to make decisions and take actions. These levels can then be predictably used by the makers and dealers to plan their strategies. With this understanding you can buy or sell stop hunts against the herd and in line with the maker moves.
THE ANATOMY OF THE HALF BATMAN PATTERN
This pattern commonly occurs at a Level I Consolidation and is similar to the Straight Away trade. Essentially, there is no need for a 2nd move back to the high because there are already traders trapped from further up and the maker does not want to provide an opportunity for them to close their trade at a profit, or even a small loss. Instead, price is moved down providing an inevitable loss for the retail traders.
The weekly pattern does not imply the use of a weekly time frame. It refers to the pattern that is seen in a 15, 60 or 240 minute chart over a period of a week.
However, makers also have seasonal variations (a new concept into cryptocurrency environment which is similar to seasonality in Forex indutry) of price movement and so it can be seen on longer time frames, though it is probably too slow to be traded effectively.
THE THREE DAYS CYCLE
A typical pattern of behaviour particularly when examining the three-day cycle is to be able to identify a peak high followed by three moves down and a reversal which forms the peak low.
Each time price moves down a level they can be referred to as achieving or making either a Level I, Level II or a Level III move.
Level I and Level II have relatively similar patterns of behaviour (1-2-3 stop hunt). However Level III tends to be choppy with a wide range and represents an area of profit-taking for the institutions and signifies the beginning of an accumulation period for another cycle.
The reasons for this behaviour can be understood if you consider what happens during the rundown:
1. On day one, you (the retail trader) are selling and the institution buys from you.
2. On day two, you are selling and again the institution buys from you.
3. However, on day three the retail trader is again interested in selling and the institution buys up heavily.
4. Now they move price up aggressively triggering stops and taking a profit. (they are using a scaling-in method to book their profits).
5. Following a Level III pullback price becomes choppy and continues because of what happens with the trader’s psychological adaptation to loss. After the market has run down for three days and retail traders have taken losses, these individuals react by pulling away from the market literally and having a few days off before coming back to trade. During this period the market is choppy and relatively stagnant until the retail traders have returned to play in the game again.
it is again worth remembering that the patterns are similar in different time frames. Additionally, the areas of reversal in both are often synchronised so that they occur at the same time in different timeframes. Using this knowledge it is possible to convert a spot trade into a swing trade when you enter it from a peak formation high peak formation low. If you are prepared to continue adding to your position as the trade progresses in your direction, it is possible to make very large profits as the whole move might be as much as 400-700 pip with numerous possibilities for supplementary entries. Another way of recalling this key issue is “The patterns are the same no matter the timeframe”.
So it is obviously important to count the levels so that you know what part of the cycle price is in at the current time. Quite obviously if you are able to enter a trade at these peak reversals then you are in a good situation to run your trade for the full extent of its journey.
So, to win in the market you should only take a short/long position when the LOD/HOD is clear. This is the only place that has a high level of certainty in directional movement.
It is also important in making these assessments to consider the bigger picture and where the market is in cycle.
This includes looking for a midweek reversal which will generally correlate with one or both of the intraday reversals.
With an awareness of the longer cycle and assuming you are in the correct place within the cycle, it is possible to convert a spot trade to a swing trade from one of the 3 day cycle peaks to the other given an appropriate entry. This would involve going from one peak formation high to the next peak low and might take several days.
On an intraday trade, it is still important to understand where you are within this larger cycle as it will help you to make a judgement about how far a run might last. For example, if price has just passed the peak high and is at a Level I accumulation then an intraday long trade after a bearish stop hunt, while valid, will not be likely to produce consistent results. Hence, it is a good idea to not take trades against the longer trend at a Level I accumulation phase.
In order to trade successfully it is important to understand clearly what part of the cycle the market is in. As previously noted there is a pattern which can be identified on both intraday and multi-day views.
Understanding the count of the intraday pattern, the 3 day pattern, and the weekly pattern is everything. If you understand these 3 things then you can expect to be better than 90 or 95% accurate.
Much of the pattern is based on exploiting the trader’s psychology. The consistent pattern of 3 moves is used to entice and encourage a particular directional behaviour. The brakes are then applied and all of a sudden fear and inexperience are exploited to have the retail traders close their positions for a loss.
The notion of counting involves an awareness of how many times a move in a particular direction has occurred and this can be observed in a number of time frames. This can include the longer time frames but for our purposes examining the cycle of 3’s on an intraday basis and over a week or more (where you can see three day cycles) is helpful.
THE COUNT OF THE THREE DAYS CYCLE
Look at the chart (a 15 min or hourly) over a week and see what is going on. For example if price is running higher and higher, there has to be a retracement.
Makers don’t have infinite amounts of capital and have to make retracements to book a profit before they continue. This is why they have aggressive pullbacks that seem to occur out of nowhere. As an example, it is possible that you were up 100 pips and all of a sudden you’re up by only 20 because they have brought price back the previous level. So what do people do, they close the trade because you don’t want to turn 300 pips into a loss and just accept the 10 or 20. However, the trade was still valid because the previous low had not been taken out and what happens next is that price takes off again, leaving the trader behind without the profit. Psychology has been used to clear traders out of the position. Price then moves aggressively up again.
Makers form zones or levels to trap traders, hit stops and book profits.
As a trader, your 1st job is to identify the zones, particularly the current place in the cycle.
THE COUNT OF THE INTRADAY CYCLE
The pattern is identical to the 3 day cycle.
However, within the 3 levels, the amount of activity of the market varies.
1. The 1st level and its correction is driven by the makers and is characterised by fast moves.
2. The 2nd level and correction is market-driven in the absence of makers support. Instead it is driven by emotional traders who enter the market. Because this is not driven by the market- makers the size of the moves tends to be smaller. This is because retail traders don’t have access to the size of trades or the coordinated effort to move price at will.
3. The 3rd level and its correction see a return of the maker to the table. This is an area of profit taking for the institution where further movement in the direction of the technical trend is encouraged before the stops are triggered. Traders can become panicked and confused. During the 3 levels makers will buy from traders to create positions. The heaviest volumes are seen at the 3rd level.
Along each of the Levels there is a corresponding level of consolidation. Most of the time this will correspond to period when stops are being triggered before the next Level is started.
The consolidation zones often involve 30 to 60 pip swings in an effort to accumulate positions and hit stops. This occurs in both directions and makes the ultimate movement to the next level easier for the maker with no buying and selling pressure being exerted by the wider marketplace.
THE ACCUMULATION PHASE
There is no specific time during the day while this first phase starts.
During this period there tends to be little activity and the market just cycles back and forth between two price points.
This occurs because Bank A will buy a quantity of currency from Bank B [1]. This causes price to rise. This is followed by bank B selling the same currency to Bank C [2]and this causes price to fall. This process goes around in circles and so the price simply oscillates back and forth.
After a while, the range begins to widen [3]. This has the effect of triggering pending orders placed by breakout traders. So positions become committed and gradually accumulate as more and more traders begin to ‘take the bait’. However, when they are triggered, price is quickly pulled away and they will often be stopped out on the other side of the range which is also widening.
THE STOP HUNT
The stop hunt involves a deliberate movement outside of the range to what will become the high or low of the day. The move usually occurs in three pushes which can be as simple as three candles though you will sometimes see a small pause in the form of a pullback in the middle of this.
The stop hunt has two main objectives:
1. Take out existing stops
2. Encourage traders to commit to positions in a direction that is opposite to where the real trend is going to be
This represents the high/low of the day (HOD/LOD). Once the HOD/LOD has been hit:
– The spread is opened up by a few pips. This allows traders orders to be triggered outside their normal boundaries and they will be holding negative positions from the outset.
– It is common to see price undergo a further period of accumulation lasting 30 to 90 minutes which encourages traders to take further positions. When there are enough positions, the price is moved in the direction of the true trend and their stops will be triggered.
– There is often a second move to the HOD/LOD though most of the time it will fail to take it out (so as to not give those who got in a profitable position to escape from). This forms the typical W or M
pattern. Note that RRT formations are also common here, but they represent W’s or M’s happening on a faster time scale.
This is the preferred point of entry for most of these trades, particularly the second leg of the M or W. It is relatively slow moving and so there should be no reason to rush or impulsively take a trade.
OTHER BEHAVIOURS AT THE HOD/LOD REVERSAL
Institutions induce traders to take the wrong direction by using sharp and aggressive moves near the high or low of the day. One of the ways of identifying that you are in the right place is that the market will seem to be quiet, in consolidation and make a sharp move out of the range, faking the so-called breakout.
If you were to take a trade in the area of the HOD/LOD you might notice that price is moving around but your position changes little. If you are looking at the price board you will see that it is flickering red and blue, with lots of changes suggesting that there is lots of activity but in fact there is little. When you see this at the right time of the day, you know that the reversal is imminent.
Another observation during this period is that the spread widens. This is done so that a broader range of orders can be collected and accumulated during this period, making it even more difficult for retail traders to take profit as they are in a losing position right from the outset. The diagram below demonstrates what happens to the spread during this period.
But these patterns do fail sometimes. This occurs when there has not been enough volume to make it worth their while to take a reversal. In these situations that price is moved to the next level to further induce positions to be taken in the wrong direction, against what is to become true trend. This is called the extended stop hunt.
THE EXTENDED STOP HUNT
The maker’s motivation is to generate a stop hunt. However, if as a result of this move the accumulation of positions is inadequate for their purposes, then the stop hunt will be extended. This means that price will be pushed beyond this Level in the direction of the technical trend in an effort to induce more traders to enter positions and build up the positions required.
Like before, this move will be comprised of 3 candles or pushes. But as before this is not necessarily the case and more or less are also possible. Again the retail trader must use their own judgement and discretion.
Therefore, if you identify that after a period of time the stop hunt has not led to a reversal then you should scratch yourself outside the trade. An appropriate period of time is 2 hours following the 2nd leg of an M or W. Whether the pair has not moved in the expected direction by this time, something is wrong (institutions are stil working the cryptocurrency pair) and they have not been able to build up enough volume to make it worthwhile to reverse the market.
THE TRUE INTRA-DAY TREND
The stop hunt is then followed by a reversal and a slower trend that continues against the faked trend toward the opposite high/low for the day.
This trend tends to move in three waves, the pause between each wave representing a new opportunity to fake out traders by reversing direction and then moving against them again (these are chances for retail traders to jump on the trend if they didn’t catch it since the beginning). These pauses are often characterised by sideways movement rather than a significant retracement though both are possible.
THE OPPOSITE LOD/HOD AND REVERSAL
Ultimately the opposite LOD/HOD will be reached and there will be another reversal. This trade is likely to return a smaller profit than the initial stop hunt reversal trade though it is still worth taking.
RETURN TO ACCUMULATION
Once the reversal has occurred, price tends back toward the centre, often not far from the starting point and recommences a new period of accumulation to the next cylce.
THE LEVELS
1. PEAK FORMATION HIGH
The highest formation on the chart is the peak. They will pull away quickly and form out the M.
2. LEVEL I AND CONSOLIDATION
After falling away from the M, price falls into a new zone and reaches a Level I Consolidation.
During the consolidation they hit stops up, hit stops down and then drop it again.
You should never trade against the Peak Formation out of Level I Consolidation.
This is also the most common place for a Straight Away Trade to develop because the makers already have what they need, which is people trapped from the previous reversal.
3. LEVEL II AND CONSOLIDATION
Price drops from the Level I Consolidation to Level II and then into a new area of consolidation (Level II consolidation).
Again, they hit the stops up, hit stops down and then drop price again to Level III.
4. LEVEL III
Having reached Level III, the objective is a little different. Price will be dropped in order to demonstrate further bearish movement by
satisfying various criteria of the traders. However, they then pull away quickly, move price up and book a profit.
Level III will appear disorganised with price chopping back and forth, usually within a wide range. If you are having difficulty identifying what level you are currently in, then identify the last Level III and this should give you a point of reference depending on whether or not price has been coming to the Level III zone from above or below and what has happened since.
Additionally if you notice that price is chopping around (in other words you are currently in Level III) then you should also be aware that a reversal is imminent.
This level often causes the formation of a head and shoulders pattern, which is a special kind of M or W formation. You should buy or sell on the 2nd shoulder.
5. PEAK FORMATION LOW
Following Level III, a new Peak Formation Low is defined and the cycle starts again. This becomes an area where you are aiming to buy with the makers, even though all of your other indicators and prior technical learning will have told you that this is still in a sell zone. So you will be buying against what you have learnt previously; against the rest of the retail world; you will be buying against the trend.
TREND ASSESSMENT
There are 2 types of trend in the market place. There is the Institutions Trend (the True Trend) and the Technical Trend.
The true trend is set by the institutions and can be reversed at any time. Understanding this key note is important because it allows you to be free of the trend bias that tends to keep people locked into wanting to trade in a fixed or single direction. The trend can move in both directions, and as the makers can change the trend at will, so should you.
Identifying where you are in the count will help project the next move. It is important not to become married to this projection, because the makers are more than capable of changing trend several times in a short period of time. These moves will often have other roles with regard to triggering stops and having traders enter on the basis of a technical trend while the True Trend turns out to be different.
The hourly chart will often remove some of the noise created by the faster 15 min chart and can be useful to review the trend. A 4 hourly chart will often highlight the peak formation highs and lows even more clearly as a pin bar or a RRT formation. But a daily chart is too slow to identify the full patterns use.
If you identify areas where the patterns all line up, for example, a 4 hourly pin bar, with an hourly RRT and a 15 min W/M then the indications are very strong for a reversal at this point.
The intraday trend usually lines up with the higher time frames. It typically begins at the stop hunt.
While trading the intraday time sessions it is still important to step back and look at the bigger picture. For example, if, on the three-day cycle you are at a Level III move then you will expect the charts to demonstrate a Technical Trend. However because of the position in the cycle, you also know that the trend should be coming to an end and a collapse and reversal is imminent.
BASIC TREND ANALYSIS WITH ALL INFORMATION AVAILABLE
Features identified in an uptrend are:
– RSI in the 80 to 40 range
– Spike bottoms
– W bottoms
– The EMA 5 and EMA 13 are positive
– Price is above the EMA 50 and EMA 200
Features identified in a downtrend are:
– RSI in the 60 to 20 range
– Spike tops
– M tops
– The EMA 5 and EMA 13 are negative
– Price is below the EMA 50 and EMA 200
1. A setup and, in this case, will include understanding where the level count is, and what your assessment of true trend direction is going to be.
2. There must be a signal present for entering or staying out
The signals are:
1. A stop hunt usually in the form of a pin formation to the high or the low
2. The development of W or M
Once you understand how the market is structured there are really only four trades that you should be looking for. These trades are better than 90% accurate and can provide enormous success. When looking for the setups it is important that the setups are clear to you and well-defined. If they do not present themselves then the object of the exercise is to move on and come back tomorrow.
So when you “turn up to the office” every day the general plan will be to:
1. Identify the peak formation high or low in the weekly views
2. Identify setups within this context, specifically looking for the most
clear setups where there is maximum opportunity for scaling in heavily and trading with high lot sizes without being concerned about losses because of the size of your stoploss and the confidence you have in the setup.
CHART SETUP
The method or process can be traded with little or no indicators on the chart. However to make things easier a number of features and indicators can be added at your discretion. These include:
– Candlestick patterns
– EMA’s 5/13/50/200
– Yesterday’s HOD/LOD marker
– Pivots
– TDI
– Maker level counts
– Sweet Spot levels
CANDLESTICK PATTERNS
The 1st and perhaps most important thing to understand about candlesticks and price action is that in the wrong market conditions they have little or no meaning. For example a hammer in the middle of the trend is relatively meaningless. A hammer at the high or low of the day has a great deal of significance.
It is also important to understand that the Candlestick pattern can only be defined at the close of the candle. The expression “backside of the hour” refers to the behaviour of candles to provide their true identity in the last part of the candlestick’s period. This is a feature controlled by the institutions and is particularly noticeable in the hourly and four hourly charts. In areas where it is their intention to convince retail traders that there will be continuation it makes sense to leave the candle as a solid green or red candle until the last moments when it is pulled back revealing itself as a spike or some other relevant shape.
The candle patterns that are most helpful are:
1. SPIKECANDLES
This description includes spike candles, “Empire State candles” and oversized candles in a 15 min chart. These candles are designed to “get you excited”, trade emotionally, and encourage you to enter the market. However, price is pulled back before the candle is closed and those traders who entered on the excitement then find themselves trapped.
These candles are most often seen in the 1st leg of a reversal set up.
But it is important to remember that price will almost always pull off very quickly.
The other place that these candles are often seen is at Level III of the three-day cycle.
2. SPINNING TOPS, HAMMERS AND INVERTED HAMMERS
3. DOJI CANDLES
4. EVENING STAR AND MORNING STAR FORMATIONS
Evening Star and Morning Star candles are often described as indecision candles. In this context they are considered to be an extension of RRT formation, simply making it a 45 minute RRT instead of a 30 minute RRT. So it is not the makrs indecision, they know exactly where they’re going.
5. RRT (RAILROAD TRACKS)
RRT trick people into going in the direction of the 1st candle. But it
is snatched away quickly on the next. They are really an anomaly of an M or W pattern. The pattern simply occurs more quickly so it is compressed into a RRT.
The same compression trick can be observed when a M/W pattern in a 15 min is viewed on an hourly or 4 hourly time frame.
6. HIGH TEST PATTERN
The high test pattern occurs at the price of yesterday’s high. For
example, the general technical trend may be demonstrating an uptrend but as price reaches yesterday’s high a reversal pattern is seen. Any of the Candlestick patterns are possible in this region and all mean the same thing. You should change direction and trade against the technical trend.
If the pattern is a double tap test, but then it fails when it closes above the high of the 1st, you do not have to wait for your stoploss to be triggered, rather you can shut it down and wait another
opportunity.
You may be able to identify a close just below the previous high and not wait for price to pull away and confirm the reversal, in other
words you take the trade on the expectation of the reversal. This will create a remarkably small stoploss and a greater profit should it move back in the intended direction. (This is an area where using a sniped entry with a faster chart such as a 250T, may be helpful.)
7. LOWTEST
The low test pattern is exactly the same as the high test pattern except that it refers to the price action and subsequent changes of trend around the previous low.
News spike candles should not be used as a point of entry. You will often observe that a news spike candle pushes up very quickly and then down quickly or vice versa. Those who straddle news spike candles will sometimes find that it work when they try this on demo accounts however, when it is used in a live market the entries are pushed to the outer extremities of the spikes ensuring that they obtain the worst possible fill. Then they quickly pull back to the other side and will be stopped out. News candles are really nothing more than a means for banks, brokers and dealers to grab your money.
EMA’ S
Moving averages, when used in the appropriate way (i.e. in the context of the makers methodology) can give:
1. A true reading of market direction
2. A reading of market momentum
3. Entry and exit signals
4. Moving support and resistance points
5 Targets for a take profit
An exponential moving average, or an exponentially weighted moving average, applies weighting factors which decrease exponentially. The weightings of old data points exponentially decrease giving much more importance to recent observations while still not discarding old observations entirely.
The specific EMA’s used are the 5, 13, 50 and 200 bar EMA’s.
– Crossovers of the EMA’s can offer entry triggers or confirmation of entry triggers when viewed in the right context, the context of the market.
– The 5 and 13 EMA’s are the signal lines.
– The 50 EMA is the balance line and shows the intraday trend
– The 200 EMA is the home base defining the longer term trend. Price always returns to home base.
Note that the 200 EMA represents the 50 EMA on the next higher timeframe. So, if you are examining the 15 min chart, the 200 EMA represents the current trend on an hourly chart. In this way you can see the hourly trend on a 15 min chart.
The 50 and 200 EMA’s are used almost universally by institutions and are even reported in public announcements. Crossovers of these EMA’s can be used as buy and sell signals.
The 5 and 13 EMA’s happen to match up the TDI used and provide responsive signals. However any other rapidly moving pair of EMA’s would achieve the same goal.
The context of the EMA’s is important. If you are in Level III the EMA’s will almost certainly be heading in the wrong direction. However, when a reversal occurs, the EMA crossover will follow and this will provide a confirmation of the direction that has been taken.
Also, when the EMA’s are spread out and Level III can be identified, it is likely that the trend will be getting ready to collapse. The EMA’s will follow rather than lead.
It is very important that you accept and understand that no indicator will have the ability to identify when a trade should be entered. The pattern and the count remain the most important features. In fact, many people who use this method do not use the indicators at all. Identifying the count and the patterns is central and is enough to trade successfully.
An exception exists though, if you are having difficulty identifying the count on a price chart because there is too much noise, then using a rapid EMA such as the EMA-5, can show the patterns quite clearly.
PREVIOUS HOD/LOD MARKERS
The high and low prices from the previous day were used by the market maker to trap volume. It is therefore significant to know how price acts at these levels the following day. These levels will often line up with other support and resistance zones.
When this occurs, it is not uncommon to see price approach the line, and throw a spike over the line. At other times price might approach but not quite reach the previous high or low. This tells you that the current price is already on the correct side. You will therefore expect the price to “bounce down” or “bounce up” as the case may be. This will most often occur around the time of the London open. You should recall that this is likely to be part of the makers aim of keeping traders trapped. If they’ve already made a high for instance, and there are positions trapped here then they will not want to push price above it again but will then approach it, perhaps even spike with an enlarged spread and pull away again.
PIVOTS
Pivot points are a widely available indicator that provide a grid based on the previous day’s range. As a grid, they can provide a clue about where today’s high or low might sit. As with other indicators, their strength really appears when there is a confluence of signals.
In effect, the pivot levels are a grid of the ADR because they are based on the high, low, and close of the previous day’s candle.
So, if the previous day’s candle was red then this indicates that today might be an M1/M3 day. (This means that price will move between the M1 and M3 pivots)
Alternately, if the previous day’s candle was green then this indicates that the day might be an M2/M4 day. (This means that price will move between the M2 and M4 pivots).
Looked at another way:
1. The HOD is more likely to be located at the M3 or M4 mid-pivot points
2. The LOD is more likely to be located at or near the M1 or M2 mid- pivot points
Not surprisingly, when you consider these points on your charts you will often find that they are located 25 to 50 points above or below the Asian range.
Similarly, if you are able to enter a trade at the HOD then it is possible that a reasonable target may be at the M1 or M2. This can help when you are following the trend and trying to avoid early exits on pullbacks. Note that understanding the count is probably more effective but the pivots provide further confirmation.
There are some situations where pivots are inaccurate. If the previous day’s range has been unusually large or unusually small then their predictive value is poor.
It can be more accurate if it is coupled with an ADR indicator which tells us the average daily trading range of the last 2 weeks. So, for example, if the ADR high lines up with M3 and then also happens to be an EMA 200 in the same place then this is a high probability area for a reversal.
As with all the indicators, the trader has to act as a filter and be able to interpret that filter in the context of information being provided by the other indicators. And, be aware that the price action or chart patterns override everything else. The indicators are only there to provide a guide.
TDI (TRADERS DYNAMIC INDEX)
The TDI is an improved version of the RSI and is really a hybrid of a number of different indicators. It incorporates a number of lines:
– A basic RSI line
– A trade signal line which provides entry signals when the RSI crosses
over and this tends to be much earlier than would be possible if you waited for the RSI to cross the midline
– A market baseline which replaces in many respects the usual midline of the RSI except that it is a dynamic line and so the crossover also
occurs earlier.
– Volatility bands which are similar to a Bollinger band but applied to the market baseline of the indicator instead of price.
The volatility bands have a number of uses:
1. They act as support and resistance lines based on the close which is much stronger
2. When the bands re-contain the RSI line after breakout, it is a sign of weakening and an impending reversal. This represents a stop hunt.
3. When viewed in the proper context, they can identify stop hunts, entries and exits
Proper use of this indicator identifies the following indicator patterns which, when looked at in the context of the price action, provide an even more accurate means of triggering trades.
The patterns are:
Shark Fin Short
Look for the following features:
– The volatility bands have been tight. They show the consolidation of the range
– The RSI line breaks out of the volatility bands. Provided the price is in the right area, then this indicates that price is in the stop hunt area.
– If it reaches an overbought state and the price action is in agreement this can be an ideal place to sell.
– The RSI line will then cross back below the signal line so that it looks like a shark fin. Entering the trade at this point would still be valid.
– The RSI then crosses the red signal line. Entering a trade at this point would still be valid.
– Note that these secondary places for entry would also represent potential places for adding to your position.
– As the trade progresses, you would normally be trying to identify the count from the price data. However you get further confirmation of this when you review the shape/pattern of the RSI in its journey back to the other side of the volatility bands.
– An exit is suggested when the RSI gets the opposite Volatility band and recrosses the Signal line again.
Again, this is only valid when it is occurring at the correct place. If it occurs during the consolidation phase for instance, it is meaningless. It is meant to be used in the same area that price action would identify the stop hunt (where the M/W pattern is).
Shark Fin Long
Is the same as the Shark Fin Short, but upside down
Scaling-in positions
Use each of the entry locations noted above:
– the appearance of the shark fin (1st)
– the Market base line cross (referred to as Blood in the Water) (2nd)
– break out of the opposite volatility band during the trend run (3rd)
CONFLUENCE OF SIGNALS
Some patterns and signals when seen in combination provide very high probability setups. Overall though you must consider:
Price action and the patterns
– M/W
– Half Batman
– Straightaway
– RRT
The count
– The weekly cycle
– The three day cycle
– The intraday cycle
The indicators
– EMA 5/13
– EMA 50/200
– TDI
– Pivots
– Sweet Spot levels
Section 8: Add-ons trading strategies
Section 9: Conclusion
Section 10: Exercises
EXERCISE 1
Identify a peak formation high and a peak formation low. Then identify all the features of the 3 tiered cycle within both on a 3 day and an intraday cycle.
Understand that they will not all looked perfect but nonetheless there are variations on the theme and being able to identify the variations is fundamentally important.
EXERCISE 2
Examine one or more cycles and put yourself behind the screen. In other words, imagine you are the institution and what you would need to do at different times to trap traders and book your own profit. You will need to consider “where the money is”, what might drive traders to behave in certain ways and then consider the methods that have been put forward including circular trading, stop hunting, testing patience, and so on.
EXERCISE 3
Choose a pair, one of the majors, go back and look at the course of 5 to 10 days of trading and try to identify the key features. The features need to include:
1. Peak formation highs
2. Peak formation lows
3. Midweek/Endweek reversals
4. Levels I, 2 and 3 with their consolidation levels
5. The intraday cycle
6. Areas where peak formation highs and lows correspond with intraday reversals
EXERCISE 4
Make a list of all of the patterns, features and characteristics that have been discussed. Create a definition or description for each of them.
EXERCISE 5
Find the expected high/low for the day on the 6 majors using Pivot calculations.
EXERCISE 6
Choose a pair and find and mark-up the divergence patterns using the RSI.
EXERCISE 7
Use TDI in the context of the patterns and mark up the entry and exit signals based on TDI
EXERCISE 8
Find and identify examples of the Straightaway Trade. Mark them out, particularly looking for previous trap moves and the levels.
EXERCISE 9
Describe the processes and patterns of a 24 hour trading cycle.
EXERCISE 10
Find stop hunts, M’s W’s, Trend Runs, NYC’s, 200 Bounce, Shark Fin Long/Short on a single pair. If you can’t see it in hindsight, you will never see it in foresight.
EXERCISE 11
Create your Checklist that relates to the trades you will be looking for, the specifics of your scanning, and how you will be taking those trades, including money management, profit targets and so on.