Unable to locate package with apt-get remove

I'm trying to remove pip (because I have pip3). I did sudo apt-get remove --purge pip, and get

Reading package lists... DoneBuilding dependency treeReading state information... DoneE: Unable to locate package pip

But which pip gives

/usr/local/bin/pip

I can confirm that pip doesn't work, because pip --version gives

bash: /usr/local/bin/pip: /usr/bin/python: bad interpreter: No such file or directory

How to I clean up my system?

FYI I'm using Ubuntu on Windows (WSL). I really hope this isn't the problem, because everything else seems to be working fine.