If you are thinking about taking advantage of low interest rates to refinance your first mortgage, you may want to do so before you take out a second mortgage. Some first-mortgage lenders require you to pay off a first and a second mortgage before they will approve the loan. In other cases, the second may change your financial picture enough that you will not qualify for a first-mortgage refinance.
If you can save significant sums by refinancing your first mortgage, do that first. That way you take advantage of the savings over the life of the loan. Then look at a second mortgage. While you may be able to get a subordination agreement (an agreement allowing you to keep an existing second mortgage while refinancing a first mortgage) from the lender who offered your first mortgage, that may not always be possible.
In short: Focus on your primary mortgage first; then look at second mortgages for improvements, debt consolidation, etc.