Borrowed from French congé, from Latin commeātus (“a leave; permission for a leave”). As an architectural term, a French calque of ἀποφυγή (apophugḗ, “a leave, an escape; an architectural feature”).
Practice speaking and memorizing "congé" and many other words and sentences in English.