We can't erase the past, so instead of trying to forget it, rejoice in God's grace that delivered you from your guilt and the power of your sin and the continuing guilt and power of your sin today. I think of Paul who always thought of himself as the chief of sinners because he had slaughtered Christians. That was what Paul was doing, so he could say, “My identity in myself, if I have to look at myself, is chief of sinners, but that's not my real identity. My real identity is in Christ, seated with him in heavenly places. While I was dead in trespasses and sins, by nature a child of wrath, he delivered me.” That's his new identity.
The Apostle Peter in 1 Peter 1 says,
And if you call on Him as Father, who judges impartially according to each one's deeds, conduct yourselves with fear throughout the time of your exile, knowing that you were redeemed from the futile ways you inherited from your forefathers. Not with perishable things such as silver or gold, but with the precious blood of Christ, like that of a lamb without blemish or spot. He was foreknown before the foundation of the world, but was made manifest in the last time for the sake of you who through him are believers in God, who raised him from the dead and gave him glory, so that your faith and hope are in God, having purified your souls by your obedience to the truth for a sincere brotherly love, love one another earnestly from a pure heart since you have been born again, not of perishable seed, but of imperishable through the living and abiding Word of God for all flesh is like grass and all of its glory like the flowers of grass, the grass withers and the flower fades, but the Word of the Lord remains forever and this Word is the gospel that was preached to you. (1 Pet. 1:17-25)
A wonderful story a pastor friend told me; he had a woman come to him and she would always cry during the service and afterward he called her and said he said, “can I talk to you, you're really struggling with something. And she said, “I can't really be forgiven.” She said, "I had an abortion and I can't get it out of my mind I'm that person who aborted her child."
He said, "You know, based on James, if you confess your sins one to another God will forgive you." He declared to her, "your sins are forgiven in Christ's name." And she says, “but what about the abortion that I had?” And he said, “What abortion?” God forgives and he really does forget even when we don't.
Adapted from an answer given on Episode 115 of the Core Christianity Radio Show.