“The day I lost you” is an adjective (edit: wait, adverb?) phrase here, not a noun phrase. But either would be grammatically correct, so you can (at most) say it’s ambiguous.