PVE in EVE is any combat situation situation where you are fighting NPCs (As opposed to PVP where you are fighting players)
Examples include Mission Running, ratting, incursions, exploration, anomalies, and fighting Sleepers in wormholes. Clearing rats from Belts, Gas Clouds, Radar sites and Magnometric Sites can also be considered PVE.
In most stations you can find agents (They have their own tab in the station interface). If you have a sufficient standing with the agent or their corporation you can initiate a conversation with them and they will give you a mission. There are several types of missions (Kill missions, Courrier missons, Trade missions and Mining missions)
Mission running is mostly done in high sec to avoid the interference from PVP which is possible in low/null sec.
NOTE: if you are in Dreddit we are currently in a state of perpetual war, Hi-Sec won't save you. Enemy corporations who have formally declared war with us are allowed to shoot you, and CONCORD (the police) will not punish them for it. Anything you've heard or read about avoiding ganks in highsec is irrelevant. Your opponent will not be killed by CONCORD, which means they can and will engage you in anything they have. Whether it's a cheap T1 frigate to hold you down and kill you slowly, or a 1 billion ISK tier 3 strategic cruiser. It doesn't matter because CONCORD won't come to save you, so they don't risk or lose anything by killing you.
More information on Mission Running can be found here.
This is going into asteroid belts or combat sites and killing the NPC pirate ships (rats) in them. You then get bounties, loot and salvage from them. Far quicker than missions. Only worthwhile in low/null sec. The lower the security status of the solar system, the bigger bounties appear.
Recommended ship: Cruiser/Battlecruiser/Battleship. Each have their own pros and cons.
More information on Ratting can be found here.