Oracle interview question

How would you modify a transformer architecture to include an external vector.