Introduction

C++ is a language derived from C, so in essence all problems at link time boil down at declaring stuff but not defining it.

Declaring something in C++ means bringing the entity into existence in the program, so it can be used after the declaration point. Defining something means giving a complete description of the entity itself. You can declare a class or a function, and it means this class and this function do exist. But to completely describe a class and a function you have to define them. A class definition provides a list of base classes of that class, a list of members (data members and member functions) of that class, etc. A function definition provides the executable code of that function. All definitions are declarations but not all declarations are definitions.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
// Defines variable 'x'
int x;
// Declares variable 'y'
extern int y;
// Declares class 'A'
struct A;
// Declares function 'f(int)'
void f(int);

// Defines class 'A'
struct A
{
    // Declares member function 'A::g(float)'
    void g(float);

    // Defines member function 'A::h(char)'
    void h(char) 
    { 
      // Code
    }

    // Defines data member 'A::x'
    int x;

    // Declares static data member 'A::y'
    static int y;
};

// Defines  function'f(int)'
void f(int)
{
 // Code
}

// Defines member function 'A::g(float)'
void A::g(float)
{
 // Code
}

// Defines static data member 'A::y'
int A::y;

C++, in contrast to C, strongly sticks to the One Definition Rule which states that entities can be defined at most once in an entire program. Of course this may not be completely true depending your own the definition of "entity": template functions when instantiated by the compiler can be defined more than once in the program, and some magic happens so this does not become a problem.

Anyway, C++ brings its own set of linking issues which may fool even the most experienced C++ developer.

Static data members are only declared inside the class specifier

Some might argue that this is one of the most common source of linking issues when using C++. Truth be told, static data members are just global variables in disguise so most people will avoid them. However, there are cases where a static data member may come in handy, for instance when implementing the singleton pattern.

The problem lies that, although usual (nonstatic) data members are defined when they are declared inside a class (like in line 23 of the example above), static data members are only declared. Thus in line 26 of the example above A::y is only being declared. Its actual definition is given in line 42. The actual definition of a static data member will go in the implementation file (usually a .cpp or .cc file).

So the usual case goes like this: you realize you need a static data member. You add it to the class. Your code compiles fine but does not link. In fact 'A::y', the static data member you just added is undefined? How can this be?

Now you know the reason.

What is the reason this issue is hit so many times? Well, there are three reasons. A historical one, where early versions of C++ compilers allowed this. A quirk in the C++ language itself where const integral and enumerator static data members can be declared and initialized in the class itself (thus defining them as well). And finally, a linguistic issue, since in Java and C# static fields are declared like any other fields plus a static specifier.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// -- Header file
class MySingleton
{
public:
    static MySingleton& getInstance()
    {
        if (singleton_ == 0)
            singleton_ = new MySingleton;
        return *singleton_;
    }
private:
    // Usual private constructor
    MySingleton() { }
    // Declaration
    static MySingleton *singleton_;
};

// -- Implementation file
// Definition
MySingleton* MySingleton::singleton_ = 0;

Not all headers are created equal

The usual myth is that C++ is a superset of C. Well, it looks like as a superset of C but they are actually two different languages. That said, they share so many thinks that interfacing C++ and C is pretty straightforward, in particular when the former must call the latter (the opposite may be a bit more challenging).

Thus, it is not unsual to see that a C++ program #includes C header files. Chances are that the headers of your operating system will be in C. Being able to #include a C header and using the entities declared in it is one of the strengths of C++. And this is the source of our second problem.

Remember that in C++ functions may be overloaded. This means that we can use the same name when declaring two functions in the same scope as long as they have different enough parameter types.

1
2
3
4
5
6
7
// Declaration of 'f(int)'
void f(int);
// Declaration of 'f(float)'
void f(float);
// Redeclaration of 'f(int)' since, in a parameter, 'const int' cannot
// be distinguished from 'int'
void f(const int);

It may be non obvious, but we cannot give these two functions declared above the same f name. So the compiler crafts an artificial name for f(int) and f(float) (this is called a decorated name or a mangled name). For instance they could be f_1_int and f_1_float (here 1 would mean the number of parameters). The C++ compiler will internally use these names when generating code and the lower levels will just see two diferent names.

But overloading cannot be applied to C. Thus we run into a problem here. If we #include C headers, the names of these functions cannot be overloaded thus a C compiler will generated code using the (undecorated) name of the function. If our C++ compiler always uses a decorated name, there will be an unresolved symbol. The C++ compiler cannot tell if this is C or C++. Can it?

Good news, it can. You can define the linkage of declarations in the code. By default linkage is C++ so overload works as described above. When you want to #include a C header, you will have to tell the C++ compiler that the linkage of the declarations is C, not C++. Most of the time you will find these lines in the beginning of a C header intended to be used from C++.

1
2
3
4
5
6
7
8
9
10
11
12
13
// Remember this is a C header so protect ourselves when this is compiled using C
#ifdef __cplusplus 
// This 'extern "C"' syntax is only valid in C++, not in C.
extern "C" { 
// From now everything has C linkage. 
#endif

/* Library declarations in C */

#ifdef __cplusplus 
// Close the brace opened above
}
#endif

Virtual member functions and virtual tables

Finally one of the, in my opinion, most confusing link errors when using a C++ compilers: virtual table unresolved references.

Virtual member functions are, in C++ parlance, polymorphic methods of other programming languages (like Java). Virtual member functions can be overridden by derived classes (descendant classes) thus when called, they must be dispatched dinamically.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
struct A
{
    virtual void vmf(float*);
    virtual void vmf2(float*);
};
struct B : A
{
    virtual void vmf(float*);
    virtual void vmf3(float*);
};

virtual B::vmf(float*)
{
    // Code
}

void g(A* pa, float *pfl)
{
  // Dynamic dispatch 
  // we don't really know if A::vmf or B::vmf will be called
  pa->vmf(pfl);

  // Static call to A::vmf since we qualified the function being called
  pa->A::vmf(pfl);

  B b;
  // Static call to B::vmf, no doubts here since the dynamic type (in memory)
  // of 'b' and its declared type must be the same
  b.vmf(pfl);

  A& ra(*pa);
  // Dynamic dispatch again
  ra.vmf(pfl);
}

Dynamic dispatch is implemented using a virtual method table (or vtable). Every class with virtual methods (called a dynamic class) has a vtable. This vtable is a sequence of addresses to member functions. Every virtual member function is assigned an index in this table and the addresses points to the function implementing the virtual member function for that class. For instance class A above has two member functions vmf and vmf2. The vtable of A, then will have two entries, 0 and 1, and will point to the functions A::vmf and A::vmf2 respectively. The vtable of B will have three entries, 0, 1, 2, that will point to functions B::vmf, A::vmf2 and B::vmf3 respectively.

Every object of a dynamic class has a hidden data member (called the virtual pointer) that points to the vtable of its class. When C++ specifies that a call goes through dynamic dispatch (in C++ parlance, a call to the ultimate overrider), we do not call directly any function but instead, through this hidden data member, we reach the vtable and using the index of the virtual member function being called, we retrieve the entry containing the addresses to the real function. Then this addresses is used in an indirect call.

Since both the virtual table and the virtual pointer are hidden from the eyes of the developer, sometimes errors in our code may cause link errors.

The compiler does not emit a vtable

This may not apply to all C++ compilers, but usually a C++ compiler only emits a vtable when it finds a definition of a virtual member function. Note that virtual member function definitions for a given class may be scattered in several files. Magic happens again so more than one definition of the vtable of a given class in several files does not become a problem at link time.

But, what if you forget to define all virtual functions? This may look contrived but in my experience this may happen by accident. The problem lies on the error at link time, which is really confusing.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
struct A
{
    int x_;
    A(int x) : x_(x) { }

    // We forget to define A::foo
    virtual void foo();
};

void quux(A* a)
{
    // Dynamic dispatch
    a->foo();
}

int main(int argc, char * argv[])
{
    A a(3);
    quux(&a);
}

If you compile and link this with g++ (I use -g since it improves link error messages by using the debugging information).

$ g++ -o prova test.cc -g
/tmp/ccl71r2A.o: In function `A':
test.cc:4: undefined reference to `vtable for A'
collect2: ld returned 1 exit status

But the line 4 is the constructor. You see now how confusing this message is, don't you? What is going on?

Well, everything makes sense if we remember that hidden data member I mentioned above, the virtual pointer. As a data member of a class it must be initialized in the constructor. It is initialized with the address of the virtual table of A. But the virtual table of A was not emitted since we forgot to define all virtual member functions. Thus, unresolved reference for the virtual table.

Missing virtual member functions in base classes

Remember that the vtable contains entries for all the virtual member functions of the base tables. The vtable is statically initialized (this is, the compiler "hardcodes" in the generated code, in the data section) the addresses of each entry. What if we forget to define a virtual member function of a base class?

Consider this example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
struct A
{
    int x_;
    A(int x) : x_(x) { }

    virtual void foo();
    // We forget to define A::foo2
    virtual void foo2();
};

void A::foo() 
{
    // Definition of A::foo
}

struct B : A 
{
    B(int x) : A(x) { }

    virtual void foo() 
    { 
        // Definition of B::foo
    }
};

void quux(A* a)
{
    a->foo();
}

int main(int argc, char * argv[])
{
    B b(3);
    quux(&b);
}

If we compile and link with g++ we get

/tmp/cc4t9NG3.o:(.rodata._ZTV1B[vtable for B]+0xc): undefined reference to `A::foo2()'
/tmp/cc4t9NG3.o:(.rodata._ZTV1A[vtable for A]+0xc): undefined reference to `A::foo2()'
collect2: ld returned 1 exit status

This happens because vtables of A and B refer to A::foo2, but we forgot to define it. Fortunately, now the error message is easier to grasp: some function is missing.

Obviously, many more link errors caused by C++ exist, but I think the ones shown here are quite common and the error messages related to them are quite confusing.

Introduction

C++ is a controversial language: you love it or you hate it. As always, knowing better about something allows one to make better arguments for or against that thing. This is what this series is about. Here I’ll explain some of the less known (except for C++ expert programmers, of course) features of C++.

Let’s start with templates, which account for such a huge part of the language.

Templates

Everyone knows that C++ has templates. Templates is an implementation of the «I have an algorithm that can be used with many different types» idea. This idea is called generic programming and is a pretty powerful one. This is why it is present in almost all modern languages.

Back to C++. C++ defines two kinds of templates: classes templates and function templates. Class templates define an infinite set of classes while function templates define an infinite set of functions. The elements of these sets of classes or functions are called specializations. Every template has a template-name which will be used to name a specific specialization.

Template declarations

Consider these two declarations

1
2
template <typename T>
struct my_list { ... }

1
2
template <typename T>
void max(T a, T b) { return a > b ? a : b; }

These are template declarations. The first one declares a class template and its template-name is my_list, the second one defines a function template and its template-name is max. A template declaration is just a declaration preceded with something without an official name that starts with template <…>, I will call it the template header (but note that this name is not blessed by the C++ standard at all, it just makes sense to me call it like this).

The template header defines what are the parameters of the template class. These are called the template parameters. A type-template parameter, like that T shown above, is a “type variable”. This is the most usual template parameter as it allows to parameterize the declaration over one or more type variables. C++, though, has two more kinds of template parameters: nontype-template parameters and (the funny named) template-template parameter. A nontype-template parameter allows us to parameterize the declaration over a (compile-time) constant integer value. Here “integer value” is a very broad term: of course it includes all integers, but also enum values (enumerators) and addresses of (statically allocated) variables and functions. A template-template parameter allows us to parameterize a declaration over another class template with appropiate template parameters.

1
2
template <typename T, int N> // N is a nontype-template parameter
struct my_fixed_array { };

1
2
template <template <typename T> MyContainer> // MyContainer is a template-template parameter
struct adaptor { };

Specializations

I said above that a class template or function template defines an infinite set of classes or function and that each element of that set was called a specialization. There is a specialization for every possible value that a template parameter can have. Such values are not bounded thus there is an infinite number of specializations (well, we could argue that constant integer values are finite in the language, but types are clearly not finite).

We give value to template parameters of a template by means of template arguments. These template arguments always appear in what is called a template-id. A template-id is just the template-name followed by a list of template-arguments enclosed in < and >.

1
2
my_list<int> l;// Here T has value int, we will write it as T ← int
max<float>(3.4f, 5.6f); // T ← float

Primary template and partial specializations

When we first declare a class template or a function template, such declaration defines the primary template.

1
2
template <typename T>
struct my_list { ... };

1
2
template <typename T>
void swap(T& a, T& b);

Class templates (but not function templates!) can have an extra set of template declarations called partial specializations. A partial specialization looks like a normal class template declaration but the template-name is now a template-id where the template-arguments partially specialize the given template parameters.

1
2
3
4
5
6
7
8
9
10
11
12
// 1) Partial specialization for "pointer to (any) P" type
template <typename P>
struct my_list<P*> { }; 

// 2) Partial specialization for "array of (any) Size of (any) Element"
template <typename Element, int Size>
struct my_list<Element[Size]> { };

// 3) Partial specialization for "pointer to function with two parameters 
// Arg1 and Arg2 returning Ret"
template <typename Ret, typename Arg1, typename Arg2>
struct my_list<Ret (*)(Arg1, Arg2)>;

A C++ compiler will always pick the partial specialization (if any) that is “closer” to the one requested in the template arguments. If no partial specialization matches, the primary template is chosen instead. The exact algorithm is not important here.

1
2
3
4
5
6
7
8
9
10
my_list<int> l0; // will pick the primary template T ← int

my_list<int*> l1; // will pick partial specialization 1) 
// where P ← int (note that respect to the primary template this is T ← int*)

my_list<int[10]> l2; // will pick partial specialization 2) 
// where Element ← int and Size ← 10

my_list<int (*)(float, double)> l3; // will pick partial specialization 3) 
// where Ret ← int, Arg1 ← float and Arg2 ← double

I think this is enough for today regarding C++ templates. More craziness to come. Stay tuned.

It's no secret that Google Mail has become, over the last years, the most widely used email server and client on the world. Not only it's basically free, but with the use of Google Apps you can even use it on your own domains.

Because so many people use it, even system administrators, it may be good to know how to use it to send system emails. Also, because Ruby is actually the only scripting language I feel comfortable with, I'll show you a simple library to send emails using Google SMTP server as an outgoing server, so you don't have to configure your server with sendmail, postfix or another typical UNIX mail server.

The first thing we will need is to include (and install) two gems:

• net/smtp (actually this comes from the Standard Library on Ruby 1.9)
• tlsmail

With those two libraries, we can already send a simple email, using the standard SMTP format:

def send_email from, to, mailtext
  begin 
    Net::SMTP.enable_tls(OpenSSL::SSL::VERIFY_NONE)
    Net::SMTP.start(@smtp_info[:smtp_server], @smtp_info[:port], @smtp_info[:helo], @smtp_info[:username], @smtp_info[:password], @smtp_info[:authentication]) do |smtp|
      smtp.send_message mailtext, from, to
    end
  rescue => e  
    raise "Exception occured: #{e} "
    exit -1
  end  
end

You can see here that the SMTP info is stored in a variable @smtp_info. We will take care of that later. Also, the variable mailtext passed to the method also needs a special format. More on that later as well. The really important fragment of code here is the one that calls enable_tls on the Net::SMTP module. This method is provided by the tlsmail gem and will allow our SMTP connection to use TLS as the authentication method. The other part of the code is pretty straightforward: we simply call the start method with a block, in which we actually send the email with send_message. Note that we have to provide the start method with the SMTP info of our Google Mail server account. This includes the server, which will be smtp.gmail.com, the port, which is 587, the HELO, which is gmail.com if using a standard account or your domain FQDN if using your own domain, and finally your username and password. For the authentication parameter we have to provide :plain (TLS will be used on top of that).

Now let's see how the mailtext string is built. In this case I'll be using a plain text format with two different variants: a simple text email, or an email with an attachment.

To send a simple text email, we have to follow the SMTP standard. I took the info from this tutorialspoint post. Here's the pattern we have to follow to build a compliant string:

def send_plain_email from, to, subject, body
  mailtext = <<EOF
From: #{from}
To: #{to}
Subject: #{subject}

#{body}
EOF
  send_email from, to, mailtext
end

Note the importance of the indenting here, as the from/to/subject lines must start at the first text column. With this simple method, you can then simply call the method we programmed before with the resulting string as a parameter and the email will be sent. Pretty easy.

Sending an attachment is a bit more complicated. As SMTP email is send in plain text, attachments are encoded in base64 strings, and are added to the message string in a special way. Here's how to do it in Ruby:

def send_attachment_email from, to, subject, body, attachment
# Read a file and encode it into base64 format
  begin
    filecontent = File.read(attachment)
    encodedcontent = [filecontent].pack("m")   # base64
  rescue
    raise "Could not read file #{attachment}"
  end

  marker = (0...50).map{ ('a'..'z').to_a[rand(26)] }.join
  part1 =<<EOF
From: #{from}
To: #{to}
Subject: #{subject}
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=#{marker}
--#{marker}
EOF

# Define the message action
  part2 =<<EOF
Content-Type: text/plain
Content-Transfer-Encoding:8bit

#{body}
--#{marker}
EOF

# Define the attachment section
  part3 =<<EOF
Content-Type: multipart/mixed; name=\"#{File.basename(attachment)}\"
Content-Transfer-Encoding:base64
Content-Disposition: attachment; filename="#{File.basename(attachment)}"

#{encodedcontent}
--#{marker}--
EOF

  mailtext = part1 + part2 + part3

  send_email from, to, mailtext
end

As you can see, in the first place the file is read and converted to a base64 string. After that, the message is generated. SMTP uses a special unique marker to delimit the attachment from the rest of the text. In here we use the line (0...50).map{ ('a'..'z').to_a[rand(26)] }.join (extracted from StackOverflow) to generate a 50 char length random string. Although it's very unlikely to happen, we should check that this random string does not appear anywhere else in the message body or the base64 converted attached file before using it as a delimiter.

After that, the rest of the message is built, specifying it has an attachment and its delimiter in the following lines:

MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=#{marker}
--#{marker}

The file is actually attached some lines below. After that, we can pass this new string to the method that sends the email, and all done.

Now, because our SMTP info can be sensitive (it contains our username and our password), it might not be a good idea to just hardcode this info in the email sending script. That's why I've used a yaml serialized hash to store this info, so we can load it at any time. Doing this is really easy with the yaml gem:

smtp_info = 
    begin
      YAML.load_file("/path/to/your/smtpinfo")
    rescue
      $stderr.puts "Could not find SMTP info"
      exit -1
    end

An example file would look like this:

---
:smtp_server: smtp.gmail.com
:port: 587
:helo: gmail.com
:username: user@gmail.com
:password: your_password_here
:authentication: :plain

Now that we have all the parts programmed, we should only pack it a little so it can be of use as a library. The following code contains a simple script with a class to send the emails and a little program that reads parameters from the command line:

require 'net/smtp'
require 'tlsmail'
require 'yaml'

class SMTPGoogleMailer
  attr_accessor :smtp_info

  def send_plain_email from, to, subject, body
    mailtext = <<EOF
From: #{from}
To: #{to}
Subject: #{subject}

#{body}
EOF
    send_email from, to, mailtext
  end

  def send_attachment_email from, to, subject, body, attachment
# Read a file and encode it into base64 format
    begin
      filecontent = File.read(attachment)
      encodedcontent = [filecontent].pack("m")   # base64
    rescue
      raise "Could not read file #{attachment}"
    end

    marker = (0...50).map{ ('a'..'z').to_a[rand(26)] }.join
    part1 =<<EOF
From: #{from}
To: #{to}
Subject: #{subject}
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=#{marker}
--#{marker}
EOF

# Define the message action
    part2 =<<EOF
Content-Type: text/plain
Content-Transfer-Encoding:8bit

#{body}
--#{marker}
EOF

# Define the attachment section
    part3 =<<EOF
Content-Type: multipart/mixed; name=\"#{File.basename(attachment)}\"
Content-Transfer-Encoding:base64
Content-Disposition: attachment; filename="#{File.basename(attachment)}"

#{encodedcontent}
--#{marker}--
EOF

    mailtext = part1 + part2 + part3

    send_email from, to, mailtext
  end

  private

  def send_email from, to, mailtext
    begin 
      Net::SMTP.enable_tls(OpenSSL::SSL::VERIFY_NONE)
      Net::SMTP.start(@smtp_info[:smtp_server], @smtp_info[:port], @smtp_info[:helo], @smtp_info[:username], @smtp_info[:password], @smtp_info[:authentication]) do |smtp|
        smtp.send_message mailtext, from, to
      end
    rescue => e  
      raise "Exception occured: #{e} "
      exit -1
    end  
  end
end

if __FILE__ == $0
  from = ARGV[1]
  to = ARGV[2]
  subject = ARGV[3]
  body = ARGV[4]
  attachment = ARGV[5]
  smtp_info = 
    begin
      YAML.load_file(ARGV[0])
    rescue
      $stderr.puts "Could not find SMTP info"
      exit -1
    end

  mailer = SMTPGoogleMailer.new
  mailer.smtp_info = smtp_info

  if ARGV[4]
    begin
      mailer.send_attachment_email from, to, subject, body, attachment
    rescue => e
      $stderr.puts "Something went wrong: #{e}"
      exit -1
    end
  else
    begin
      mailer.send_plain_email from, to, subject, body
    rescue => e
      $stderr.puts "Something went wrong: #{e}"
      exit -1
    end
  end
end

And that's all! You can use the script as a standalone command to send an email with some command line arguments or just require it in your ruby script and use the class to send the messages.

Ruby on Rails performance is a topic that has been widely discussed. Whichever the conclusion you want to make about all the resources out there, the chances you'll be having to use a cache server in front of your application servers are pretty high. Varnish is a nice option when having to deal with this architecture: it has lots of options and flexibility, and its performance is really good, too.

However, adding a cache server in front of your application can lead to problems when the page you are serving has user dependent content. Let's see what can we do to solve this problem.

Some days ago I learnt about The FizzBuzz Test and did a simple implementation in Ruby. The FizzBuzz test is a simple algorithm that is supposed to do the following:

For each number from 1 to 100:

• If the number is divisible by 3, print "Fizz"
• If the number is divisible by 5, print "Buzz"
• If the number is divisible by both 3 and 5, print "FizzBuzz"
• Otherwise print the number

I was just reading about how you can use the Enumerator class to have generators in the Programming Ruby 1.9 book, and thought that a good implementation could be done using just an Enumerator, so here it is, along with a simple RSpect test:

FizzBuzz = Enumerator.new do |yielder|
  count = 1
  loop do
    if count % 3 == 0
      if count % 5 == 0
        yielder.yield "FizzBuzz"
      else
        yielder.yield "Fizz"
      end
    elsif count % 5 == 0
      yielder.yield "Buzz"
    else 
      yielder.yield count
    end
    count += 1
  end
end

require_relative 'fizzbuzz'

describe FizzBuzz do
  before(:all) do
    @fizzbuzzes = FizzBuzz.first(100)
  end

  it "returns 'Fizz' for all multiples of 3" do
    @fizzbuzzes[3-1].should == 'Fizz'
  end

  it "returns 'Buzz' for all multiples of 5" do
    @fizzbuzzes[5-1].should == 'Buzz'

  end

  it "returns 'FizzBuzz' for all multiples of 3 and 5" do
    @fizzbuzzes[60 - 1].should == 'FizzBuzz'

  end

  it "returns the passed number if not a multiple of 3 or 5" do
    @fizzbuzzes[1 - 1].should == 1
  end
end

You can also find the code in its GitHub repository: https://github.com/brafales/ruby-fizzbuzz.