Distributed Telemetry with Golang, React, and Sentry

Distributed Telemetry with Golang, React, and Sentry

Building robust developer-friendly distributed services

First thing's first!

What is Telemetry?

From Stackify's blog post,

Telemetry is the automatic recording and transmission of data from remote or inaccessible sources to an IT system in a different location for monitoring and analysis.

So, deploying solutions with telemetry tools built-in allows us to monitor how an app performs, how long certain actions take, and most importantly, when things do go wrong(and eventually, they will), well-implemented telemetry proves essential in finding out exactly what happened fast!

It is one of those things you might not want to integrate into everything you create. A portfolio website, for example, is probably not worth the effort of integrating telemetry into, but dealing with large amounts of user data is a much more sensitive task and would definitely warrant some additional security measures and investment.

Sentry and Golang

Sentry is a distributed telemetry platform. The keyword here is distributed. Apart from just helping monitor independent services or applications separately, Sentry builds in a structure that allows the platform to stitch together related user instances from different sources and provide a very convenient bird's eye view of exactly what checkpoints a user session passed through, along with the application state at that point.

Let's implement it

To understand it better, we can now do a quick test project.

At the end of the following instructions, you should have something like the below graphic:

Sentry Trace

If you look carefully, you'll be able to see the step where the browser calls our API, and sentry stitches together the backend spans as well, to give you a complete performance outlook of all your services in one place. I am referring to the part shown below:

Sentry frontend-backend tracing

First, head on to Sentry.io and register a free account.

Once logged in, create two projects in your new team. The free tier should suffice. One of these projects is going to be used for our react application, while the other one will be used for the Go backend server. You can name and choose the configurations appropriately.

Keep a note of the DSN for both of the projects. You can find them in the project setting, by clicking on the "Client Keys" options from the Settings sidebar menu, like below:

DSN keys You want the one on top!

Next, we can set up both our projects. If you wish to directly look at the code(with comments), you can follow this link to the Github repository.

We want to create a basic go server and the basic create react app template to test our tooling setup.

So, make sure you have go and Node setup. Yarn is optional.

Create the react app with

npx create-react-app

and create a new go repo with

go mod init

For the react app, the setup is very basic. We just want to initialize Sentry, and add a useEffect with our network request. Sentry takes care of the rest.

So, add the below in index.js, above the ReactDOM.render call

Sentry.init({
  dsn: process.env.REACT_APP_SENTRY_DSN, // Retreiving the DSN from .env file
  integrations: [new Integrations.BrowserTracing()], // Enabling automatic browser tracing
  tracesSampleRate: 1.0, // Capturing all events
});

Then, in App.js, let's add the network request we wish to monitor. Add a useEffect block in the App functional component, as below:

useEffect(() => {
    const url = `${process.env.REACT_APP_API_URL}getPosts`; // get API host from .env file
    fetch(url)
      .then(res => res.json())
      .then(res => {
        console.log({
          res
        });
      })
      .catch(reason => {
        console.error({
          reason
        })
      })
  }, [])

The above effect now runs once on pageload of the app. This is all for the setup in our frontend. Sentry automatically adds its markers in the network requests and records them.

Next, we will be adding sentry to our GIN backend. Next, will be just a lot of code for our GIN application. This includes our router and one controller. Once the GIN app is done, we add the sentry code.

The backend will only be used to get the 10 latest stories from Hackernews, retrieve their info one by one using their ids, and return all of the stories to the frontend.

The go.mod should look something like the below with the following direct dependencies:

module yourginserverproject

go 1.17

require (
    github.com/getsentry/sentry-go v0.12.0
    github.com/gin-gonic/contrib v0.0.0-20201101042839-6a891bf89f19
    github.com/gin-gonic/gin v1.7.7
    github.com/joho/godotenv v1.4.0
)

Add the dependencies to your go.mod and run the following command:

go mod download

Next, add main.go in the project root:

package main

import (
    "log"
    "os"

    "yourginserverproject/controllers"

    "github.com/gin-gonic/gin"

    "github.com/gin-contrib/cors"
)

func main() {
    r := GetRouter()
    r.Use(gin.Logger())
    if err := r.Run(":8090"); err != nil {
        log.Fatal("Unable to start:", err)
    }
}

func GetRouter() *gin.Engine {
    r := gin.Default()

    corsConfig := cors.DefaultConfig()
    corsConfig.AllowAllOrigins = true
    corsConfig.AllowHeaders = append(corsConfig.AllowHeaders, "*") // For test project

    r.Use(cors.New(corsConfig))

    r.GET("/getPosts", controllers.GetPostsHandler)

    return r
}

The above starts a very basic GIN server with no routes/controllers. Now, we add one controller in controllers/getPosts.go that services out fetch request from the react application:

package controllers

import (
    "context"
    "encoding/json"
    "fmt"
    "io/ioutil"
    "net/http"
)

type GetPostsResponseItem struct {
    Title       string `json:"title"`
    Description string `json:"text"`
    Type        string `json:"type"`
}

type GetPostsResponse []GetPostsResponseItem

func getPostsCall() GetPostsResponse {
    var res GetPostsResponse

    // Start first HTTP request
    httpClient := &http.Client{}
    req, err := http.NewRequest("GET", "https://hacker-news.firebaseio.com/v0/newstories.json?orderBy=\"$key\"&limitToFirst=10", nil)
    if err != nil {
        fmt.Print("Error 1:", err.Error())
    }
    req.Close = true
    resp, err2 := httpClient.Do(req)
    if err2 != nil {
        fmt.Print("Error 2:", err2.Error())
    }
    body, err3 := ioutil.ReadAll(resp.Body) // JSON content in byte-array
    resp.Body.Close()
    if err3 != nil {
        fmt.Print("Error 2: ", err3.Error())
    }
    // End HTTP request

    var ids []int
    json.Unmarshal(body, &ids) // Get IDs for top stories

    fmt.Print(ids[0:10])

    for i := range ids {
        id := ids[i]

        // Get meta data for every story
        urlLink := fmt.Sprintf("https://hacker-news.firebaseio.com/v0/item/%v.json", id) 
        iResp, err := http.Get(urlLink)
        if err != nil {
            continue
        }

        resB, _ := ioutil.ReadAll(iResp.Body)
        var temp GetPostsResponseItem
        json.Unmarshal(resB, &temp)
        res = append(res, temp)
    }

    return res
}

func GetPostsHandler(c *gin.Context) {
    response := getPostsCall()

    c.JSON(200, response)
}

Now, all that is left for our application to start working(without sentry; for now) is to add a .env configuration file to the react projects to connect it to the backend.

Add a .env file to the root of the react project with the content below:

REACT_APP_API_URL=http://localhost:8090/

Replace localhost:8090 if you're running the GIN app on a different port/host. You should now be able to run the app and the server and check the browser console for the output.

Time to add the sentry essentials to our GIN application.

Modify the main.go file as below

Add the following imports:

import (
    sentry "github.com/getsentry/sentry-go"
    sentrygin "github.com/getsentry/sentry-go/gin"
    "github.com/joho/godotenv"
)

In the GetRouter function, add the sentry middleware right after initialization or the variable r:

r.Use(sentrygin.New(sentrygin.Options{}))

and finally, for main.go, add the sentry initialization code at the top of the main function:

    godotenv.Load()
    sentryDsn := os.Getenv("SENTRY_DSN")
    err := sentry.Init(sentry.ClientOptions{
        Dsn:              sentryDsn,
        TracesSampleRate: 1,
        Debug:            true,
    })
    if err != nil {
        log.Fatalf("sentry.Init: %s", err)
    }

The first line will load the variable from the project .env file to the environment, where we can pull the Sentry DSN variable from. The next few lines initialize the SDK itself.

Now, adding the sentry goodies to our controller itself. Sentry with Golang is more hands-on, of course. We change it as follows:

  1. Add sentry.captureException(err) in all of the error blocks.
  2. Add a context.Context type parameter in all of the nested functions to pass the parent's Sentry context.
  3. Add Sentry span initialization to all the functions to monitor full time taken by the functions themselves.
  4. Manage spans for all the external communication. In this case, that refers to the HTTP requests. They might be Database calls in your application.

Once that is done, the file will look like below(comments added):

package controllers

import (
    "context"
    "encoding/json"
    "fmt"
    "io/ioutil"
    "net/http"

    "github.com/getsentry/sentry-go"
    "github.com/gin-gonic/gin"
)

type GetPostsResponseItem struct {
    Title       string `json:"title"`
    Description string `json:"text"`
    Type        string `json:"type"`
}

type GetPostsResponse []GetPostsResponseItem

func getPostsCall(sentryCtx context.Context) GetPostsResponse {
    // sentry span init for function, set parent to context from calling function
    defer sentry.Recover()
    span := sentry.StartSpan(sentryCtx, "[GetPostsCall]")
    defer span.Finish()

    var res GetPostsResponse

    httpClient := &http.Client{}
    span2 := sentry.StartSpan(span.Context(), "[HTTP] Get latests stories") // Start nested span to monitor first http request
    req, err := http.NewRequest("GET", "https://hacker-news.firebaseio.com/v0/newstories.json?orderBy=\"$key\"&limitToFirst=10", nil)
    if err != nil {
        fmt.Print("Error 1:", err.Error())
        sentry.CaptureException(err)
    }
    req.Close = true
    resp, err2 := httpClient.Do(req)
    if err2 != nil {
        fmt.Print("Error 2:", err2.Error())
        sentry.CaptureException(err2)
    }
    body, err3 := ioutil.ReadAll(resp.Body)
    resp.Body.Close()
    span2.Finish() // Request workflow finished; end span
    if err3 != nil {
        fmt.Print("Error 2: ", err3.Error())
        sentry.CaptureException(err3)
    }

    fmt.Printf("Body : %s", string(body))

    var ids []int
    json.Unmarshal(body, &ids)

    fmt.Print(ids[0:10])

    span3 := sentry.StartSpan(span.Context(), "[HTTP] Get data for stories") // Start span to monitor for loop execution time
    for i := range ids {
        id := ids[i]

        urlLink := fmt.Sprintf("https://hacker-news.firebaseio.com/v0/item/%v.json", id)
        iResp, err := http.Get(urlLink)
        if err != nil {
            sentry.CaptureException(err)
            continue
        }

        resB, _ := ioutil.ReadAll(iResp.Body)
        var temp GetPostsResponseItem
        json.Unmarshal(resB, &temp)
        res = append(res, temp)
    }
    span3.Finish()

    return res
}

func GetPostsHandler(c *gin.Context) {
    // Sentry transaction initialization
    defer sentry.Recover()
    span := sentry.StartSpan(c, "[Getposts]", sentry.TransactionName("GETPOSTS"), sentry.ContinueFromRequest(c.Request)) // Take trace id from request header
    defer span.Finish()

    response := getPostsCall(span.Context()) // Call nested function; Could be DAO methods

    c.JSON(200, response)
}

Now, all that is left is to add/update the .env files

Update the .env file for the React project with your Sentry React project's DSN like below:

REACT_APP_API_URL=http://localhost:8090/
REACT_APP_SENTRY_DSN=https://dsn.sentry.io/123456

And add a .env file to the GIN project with your Sentry Go project DSN like:

SENTRY_DSN=https://dsn.sentry.io/123456

We are finally ready. Now, run both the projects and open the web app.

Keep the page open and wait a few minutes for Sentry to stitch together the frontend and backend transactions. Then, just open the Backend project in sentry and click on View Transactions. Click on the GETPOSTS transaction. You should see your new transaction here:

Transcations

Click on one and you'll see the backend transaction. Now, click on Parent from the top right box:

Click on Parent

Now, you should see something resembling the first screenshot from this post. And that is how you add distributed tracing to your React WebApp with a GIN backend!

Started writing this with the intention of publishing this within Trell's blog during my internship. Got a little late for that, but better late than never I guess! :)