Adding a toolbar button in a Firefox extension is simple. Since the Firefox user interface is described using XUL, it is enough to inject a snippet of XUL into the "toolbar palette" in browser.xul. Traditionally this would be done with a XUL overlay. Experienced XUL developers know that a button added like this won't show up in the toolbar unless it is added explicitly by the user (using View/Toolbars/Customize...). This can be solved by adding the button to the toolbar manually and saving the toolbar's "currentset", which contains the list of buttons it contains. This should be done on first run only to ensure that the user's settings are preserved if they remove the button later.
But what if you are writing a bootstrapped extension? XUL overlays are not supported, so you have to add the button to the toolbar palette by hand. Coordinating the toolbar palette, the visible toolbar and the toolbar's persistent current set isn't rocket science, but it is a bit tricky. You want to make sure that the button is added to the toolbar on first run and is visible on subsequent runs. At the same time, it should be possible to remove it and add it back using the toolbar customization feature.
This function ensures that the button is always added to the palette (i.e. what the overlay would normally do). On first run, it adds it to the far-right of the toolbar. On subsequent runs, it inserts the button manually into the correct position if it is in the currentset. This is the only bit that is substantially different from the behavior in a traditional extension. I suspect this is necessary because the button is not in the palette yet at the point when Firefox processes the currentset and adds the buttons automatically.
Remember that you should remove the button from the palette and toolbar manually when the extension is disabled or uninstalled to avoid memory leaks.