Usually up is for higher temperatures and down is for lower.
The optimal setting is dependent on the ambient temperature and how many times it is being opened during the day. Temperature fluctuations are also dependent on how full it is. The items in a refrigerator help retain the cold.
To ensure that your refrigerator is doing its job, it’s important to keep its temperature at 40 °F or below; the freezer should be at 0 °F. Since few refrigerator controls show actual temperatures, using an inexpensive freestanding appliance thermometer will allow you to monitor the temperature and adjust the setting of the refrigerator and/or freezer if necessary. Buy one for the fridge, one for the freezer, and check them often.