If your wanting 4GB, yes you will need an upgrade of power supply, because the current power supply you have right now may not support a new number of GB, and for your sake I would not risk your upgrade process failing due to too low power supply.
Hope this helps!!
