It is a long-standing problem in general relativity that the notion of angular momentum of an isolated system has supertranslation ambiguity. In this paper, we argue that the ambiguity is essentially because of the gravitational wave memory. When properly subtracting the memory effect of the observer, one can introduce a supertranslation invariant definition of the angular momentum at null infinity.