By default when you mark a task as "Finished" it will stay on your To Do List. This is so you can review your finished tasks later if you need to. You can then clear all of your finished tasks at once later.
However, there is an option to have any task that you mark as "Finished" automatically deleted. To learn more about how to turn this feature on follow the steps in the image below: