I need the following, when the page is loaded must show one open accordion with minus icon, and when closed the plus icon should change.

Here's an example in fiddle

$('div.accordion-body').on('shown', function () {
    $(this).parent("div").find(".icon-plus")
                         .removeClass("icon-plus")
                         .addClass("icon-minus");
});

$('div.accordion-body').on('hidden', function () {
    $(this).parent("div").find(".icon-minus")
                         .removeClass("icon-minus")
                         .addClass("icon-plus");
});

How to make to display open accordion with default icon minus?

There are much easier ways to do this than the current answers. You can do this all with CSS so you don't need to go into the DOM and dynamically change any elements.

If you include the following CSS (adapted from show collapse state with Chevron icon):

<!-- language: lang-css -->
.accordion-heading a:after {
    font-family: 'Glyphicons Halflings';
    content: "\2212"; /* Plus */   
    float: right; 
    color: grey; 
}
.accordion-heading a.collapsed:after {
    content: "\2b";  /* Minus */
}

Then you can even get rid of the icon elements as this will create an icon and toggle it appropriately based on whether the anchor element has the collapsed class.

Demo in jsFiddle


Or, if you don't like using psuedo selectors, you can just include both icons and then toggle their visibility as needed.

<!-- language: lang-html -->
Title One
<i class="icon-plus"></i>
<i class="icon-minus"></i>
<!-- language: lang-css -->
.accordion-heading .icon-plus { display: none; }
.accordion-heading .icon-minus { display: block; }
.accordion-heading .collapsed .icon-plus { display: block; }
.accordion-heading .collapsed .icon-minus { display: none; }

Demo in jsFiddle