Knowledge Graphs (KG) are widely used for knowledge representation. Recently, approaches aimed to represent the KG structure in an embedded space have become increasingly popular for their ability to capture high-level similarities between entities and relations. However, these embedded representations commonly give low consideration to the time aspect. Real-world entities exist and act in a defined temporal interval, consequently time is a valuable element of information in their description. In this work, we study the influence of time on the embedded representations of entities that are generated from text. The preliminary evaluation shows that generating a specific representation for temporal entities (e.g., years) can result in a more informative entity representation space. Then, we propose a new time-aware similarity metric that can be used to evaluate the similarity between entities by either flattening their time distance or boosting it.