I started using Telegram a few years ago. Most of the time I don’t use it much to have 1 to 1 conversations but rather chat in a small group of friends which I’ve known for a while now. Every now and then, we share some links to Twitter on that group, and unfortunately the Telegram official clients preview mode don’t support previewing Twitter messages with more than one image.

Take this test message as an example:

The moment I link this to a Telegram chat, this is the result:

Which is not ideal, as sometimes the message doesn’t make much sense when only one of the images is displayed.

Turns out I’ve been learning some Go over the last few months as well, so I wondered if I could write a small Telegram Bot to help me with that. I needed something to which I could send a Twitter link, and gave me back either the default Twitter preview in Telegram, or a custom made one with all the images of the message, in case there were more than one.

Introducing the new Bot

I started this more to learn a bit more of Go rather than to learn much about the Telegram and Twitter APIs, so I decided to use already built packages to deal with both the Telegram Bot API and the Twitter REST API. The following flow chart describes in a nutshell what the bot does:

 

The way the bot code is structured is through the following packages:

  • configuration: holds general bot configuration options
  • twitter: deals with the Twitter REST API, mostly though go-twitter
  • message: builds messages that telegram-bot-api understands
  • main: entry point. Initialises all the things and controls the main program flow

The main program runs as follows:

package main

import (
	"fmt"
	"log"
	"net/http"
	"os"

	"github.com/brafales/piulades-bot/configuration"
	"github.com/brafales/piulades-bot/message"
	"github.com/brafales/piulades-bot/twitter"
	"github.com/go-telegram-bot-api/telegram-bot-api"
)

func main() {
	config, err := configuration.New()
	fail(err)

	bot, err := tgbotapi.NewBotAPI(config.BotKey)
	fail(err)

	twitterClient := twitter.NewClient(config.TwitterAPIKey, config.TwitterAPISecret)

	_, err = bot.SetWebhook(tgbotapi.NewWebhook(config.CallbackURL))
	fail(err)

	updates := bot.ListenForWebhook("/")
	go http.ListenAndServe(":"+os.Getenv("PORT"), nil)

	for update := range updates {
		err = processUpdate(update, bot, twitterClient, config.ChatID)
		if err != nil {
			fmt.Printf("%v\n", err)
		}
	}
}

We set up a new configuration, which will hold some information that we will need throughout the rest of the program (more on how the configuration reads its values later). We then create a new Telegram Bot using the supplied bot key, and lastly we do the same with a Twitter client, supplying the API key and secret needed when setting up Application Only Auth.

After the initialisation code, we set up the bot in Webhook mode. The line updates := bot.ListenForWebhook("/") will return a new channel of Telegram Updates (think of them as Telegram messages sent to the bot) that will be updated by an http.Handler listening to requests made to /. After this, we can range over the updates, passing them to the method processUpdate, which will do all the heavy work.

Processing the messages

The processUpdate method reads as follows:

func processUpdate(update tgbotapi.Update,
	bot *tgbotapi.BotAPI,
	twitterClient *twitter.Client,
	chatID int64) error {
	statusID, err := twitter.GetStatusID(update.Message.Text)
	if err != nil {
		return err
	}

	tweet, err := twitterClient.GetTwit(statusID)
	if err != nil {
		return err
	}

	images, err := tweet.ExtendedEntities()
	if err != nil {
		return err
	}
	messages, err := message.Build(chatID,
		update.Message.From.UserName,
		images,
		tweet.PrintableText(update.Message.From.UserName))
	if err != nil {
		return err
	}

	for _, message := range messages {
		_, err := bot.Send(message)
		if err != nil {
			return err
		}
	}
	return nil
}

It basically follows the general flow chart shown above:

  1. We try to get a Twitter Status ID from the message the bot has received, and return immediately with an error if we can't
  2. If we find a proper ID, we go and talk to Twitter to get all the information we need about the Tweet. Again, if we can't do that, we just return with an error
  3. We try to retrieve the Tweet Extended Entities (which is the name the Twitter API gives to the multiple images you can attach to a Tweet), returning an error if something goes wrong in the process.
  4. We send the message Build to the message package with the retrieved Entities. This will get us a slice of messages that are ready to be sent to Telegram through our bot.
  5. For each one of the messages returned by the message builder, we send the Send message to our Telegram Bot, which will ensure that the message reaches its destination.

A few of these steps are worth digging into, so let’s go and do that.

Talking to Twitter

Twitter uses OAuth to manage authentication. In our case we are writing an application that needs to access Twitter Statuses (the name Twitter gives to a Tweet). For our bot, we rely on the go-twitter package, which leverages the oauth2 Go packages to handle the authentication layer. A twitter package inside our application manages all this complexity for us:

package twitter

import (
	"errors"
	"fmt"
	"net/http"
	"regexp"
	"strconv"

	"io/ioutil"

	twitterAPI "github.com/dghubble/go-twitter/twitter"
	"golang.org/x/oauth2"
	"golang.org/x/oauth2/clientcredentials"
)

//Client allows you to communicate with the Twitter API
type Client struct {
	client *twitterAPI.Client
}

//NewClient returns a new Client instance.
//apiKey is the twitter API key and apiSecret is the
//twitter API secret
func NewClient(apiKey string, apiSecret string) *Client {
	config := &clientcredentials.Config{ClientID: apiKey,
		ClientSecret: apiSecret,
		TokenURL:     "https://api.twitter.com/oauth2/token"}
	httpClient := config.Client(oauth2.NoContext)
	newClient := Client{client: twitterAPI.NewClient(httpClient)}
	return &newClient
}

We expose a Client type and a method NewClient that will create the internal Twitter client that go-twitter provides.

Apart from a client, we also expose our own Tweet type, and provide a few utility methods to create them and ask useful questions. Amongst those utility methods, we have one that gets us the Extended Entities for a given Tweet, one that gives us the original Tweet URL, and one that will provide a pretty printed version of the Tweet contents for our Telegram messages:

//Tweet holds information about a specific twitter status
type Tweet struct {
	apiTweet        *twitterAPI.Tweet
	imageDownloader ImageDownloader
}

//NewTweet returns a new instance of Tweet from a low level API response
func NewTweet(apiTweet *twitterAPI.Tweet, imageDownloader ImageDownloader) Tweet {
	return Tweet{apiTweet: apiTweet, imageDownloader: imageDownloader}
}

//GetTwit returns a Tweet associated to the statusID
func (t Client) GetTwit(statusID int64) (Tweet, error) {
	apiTweet, _, err := t.client.Statuses.Show(statusID, nil)
	if err == nil {
		return NewTweet(apiTweet, DefaultImageDownloader{}), nil
	}
	return Tweet{}, err
}

//PrintableText returns a string with the tweet information
func (t Tweet) PrintableText(user string) string {
	return fmt.Sprintf("Tweet enviat per [%s]: https://twitter.com/%s/status/%d - %s",
		user,
		t.apiTweet.User.ScreenName,
		t.apiTweet.ID,
		t.apiTweet.Text)
}

//URL returns the original tweet URL
func (t Tweet) URL() string {
	return fmt.Sprintf("https://twitter.com/%s/status/%d",
		t.apiTweet.User.ScreenName,
		t.apiTweet.ID)
}

//ExtendedEntities returns a slice with all the raw images of the Tweet
func (t Tweet) ExtendedEntities() ([][]byte, error) {
	if t.apiTweet.ExtendedEntities != nil && len(t.apiTweet.ExtendedEntities.Media) > 1 {
		entities := make([][]byte, len(t.apiTweet.ExtendedEntities.Media), len(t.apiTweet.ExtendedEntities.Media))
		for i, entity := range t.apiTweet.ExtendedEntities.Media {
			entityData, err := t.imageDownloader.Get(entity.MediaURL)
			if err != nil {
				return entities, err
			}
			entities[i] = entityData
		}
		return entities, nil
	}
	return make([][]byte, 0, 0), nil
}

Finally, the package also has the method that will look for Twitter links for a given text string, making use of Go regular expressions:

//GetStatusID gives you the twitter status id for a given text
func GetStatusID(text string) (int64, error) {
	r := regexp.MustCompile(`https?://(www\.)?twitter\.com/\w+/status/(?P\d+)`)
	matches := r.FindStringSubmatch(text)
	if matches != nil {
		id, err := strconv.ParseInt(matches[len(matches)-1], 10, 64)
		if err == nil {
			return id, nil
		}
	}
	return 0, errors.New("Could not find a twitter status id")
}

Building the messages for Telegram

As described at the flow chart, we have essentially 2 use cases here:

  1. The Tweet does not have more than one Extended Entity
  2. The Tweet has 2 or more Extended Entities

In case number (1) the current Telegram preview is good enough for us, so the message builder will simply send the same link again.

Case number (2) is more interesting, as we want the bot to be sending the multiple Extended Entities as inline messages, and also send the Tweet text (but disabling the web preview flag, so we don’t send duplicate information).

The Telegram Bot API allows us to do this in a relatively easy way:

package message

import (
	"fmt"

	"github.com/go-telegram-bot-api/telegram-bot-api"
)

//Build returns a list of Chattable objects to send via Telegram
func Build(chatID int64, user string, images [][]byte, text string) ([]tgbotapi.Chattable, error) {
	imageCount := len(images)
	if imageCount > 0 {
		chattables := make([]tgbotapi.Chattable, imageCount+1, imageCount+1)
		msg := tgbotapi.NewMessage(chatID, text)
		msg.DisableWebPagePreview = true
		chattables[0] = msg
		for i, image := range images {
			file := tgbotapi.FileBytes{
				Name:  fmt.Sprintf("Image %d", i),
				Bytes: image,
			}
			msg := tgbotapi.NewPhotoUpload(chatID, file)
			chattables[i+1] = msg
		}
		return chattables, nil
	}
	msg := tgbotapi.NewMessage(chatID, text)
	return []tgbotapi.Chattable{msg}, nil
}

Configuring our application

Last but not least, our application requires of some information to run. In our case:

  • Twitter authentication information
  • Telegram Bot authentication information
  • Telegram Chat where we want to send our messages to
  • What HTTP port should our web server listen to (needed for heroku deploys)
  • What URL should Telegram send the messages sent to the bot (the webhook or callback URL)

Because I decided to deploy the bot to heroku, I went the environment variable route to get all the sensitive data the application needs. Through heroku, you can make certain values available as OS environment variables, which our application can access. This is how I made my configuration package load all the values the application needs to run properly:

package configuration

import (
	"errors"
	"os"
	"strconv"
)

//Configuration holds application configuration
type Configuration struct {
	TwitterAPIKey    string
	TwitterAPISecret string
	ChatID           int64
	Port             string
	BotKey           string
	CallbackURL      string
}

//New returns a new Config
func New() (*Configuration, error) {
	chatID, err := strconv.ParseInt(os.Getenv("CHAT_ID"), 10, 64)
	if err != nil {
		return &Configuration{}, errors.New("Could not get chat id from environment")
	}
	return &Configuration{
		TwitterAPIKey:    os.Getenv("TWITTER_API_KEY"),
		TwitterAPISecret: os.Getenv("TWITTER_API_SECRET"),
		Port:             os.Getenv("PORT"),
		BotKey:           os.Getenv("BOT_KEY"),
		CallbackURL:      os.Getenv("CALLBACK_URL"),
		ChatID:           chatID,
	}, nil
}

The result

Once the bot has been deployed, a message can be sent to it with a link to Twitter, and the bot will proxy the message with all the needed images to the channel I’m interested to:

Which makes a lot more sense for Tweets with multiple images.

Find all the code in the piulades-bot Github repository.