Abstract: We introduce Dolores, a new knowledge graph embeddings, that effectively capture contextual cues and dependencies among entities and relations. First, we note that short paths on knowledge graphs comprising of chains of entities and relations can encode valuable information regarding their contextual usage. We operationalize this notion by representing knowledge graphs not as a collection of triples but as a collection of entity-relation chains, and learn embeddings using deep neural models that capture such contextual usage. Based on BiLSTMs, our model learns deep representations from constructed entity-relation chains. We show that these representations can be easily incorporated into existing models to significantly advance the performance on several knowledge graph tasks like link prediction, triple classification, and multi-hop knowledge base completion.