Quantitative QC: Extended BOLD QC via XNAT (CBSCentral)
Qualitative QC: Looking at your data
Preventive QC: at the scanner
Click here for slides presented by Natasha Hansen (firstname.lastname@example.org) at our workshop by her in Feb., 2013. The workshop slides give a comprehensive overview of quality control for MRI data including why QC is important, the kinds of QC: quantitative vs qualitative, quantitative QC measures in XNAT, and qualitative QC with artifact examples and unusual cases. Don't miss out on the accompanying manual on qualitative QC below!
If you have your epi data in CBSCentral labeled as type BOLD, then the extended bold QC pipeline will automatically be run on your data, shortly after it is archived. The type of your scan is set when you archive your data. See the image below for an example. If you previously used the scan type to specify study specific information, like bold_run1, you will need to move that information to the notes field.
This pipeline converts the dicoms to nifti format, creates mean, standard deviation, and SNR maps, calculates slope and mean slice intensity changes and then performs motion correction (with mcflirt from FSL) to get measures of movement.
The results will be uploaded to CBSCentral separately for each subject and each run. You can get to the BOLD QC data by clicking on the link for the run you are interested under the Assessments heading at the bottom of the session info page.
This will show you the extended BOLD qc page that looks like:
The parameter’s box on the left side has a lot of useful statistics. This contains several measures the Buckner lab has found to be useful in determining whether a dataset is usable or not. These have been developed by looking at many datasets over the years and are rough guidelines that work specifically for Siemens 3T 12 channel data from adults. To get a description of what a parameter is, hold the mouse over it.
The Slice SNR measures the timecourse SNR (mean/stdev) averaged across each slice, and then averages all the slices together. This serves as a general ‘goodness’ measure. If it is greater than 150 (for Siemens 3T 12ch adult data), then the data is good, while less than 99 is bad. This number can be low for several reasons, including subject motion and scanner artifacts. A low value should be a red flag to look more closely at the data.
Max absolute motion is also useful for determining usability of data. Values less than 1.49 are good, while greater than 2mm is bad.
Number of movements greater than .5mm should be less than 5 for good data, greater than 5 is bad. This is calculated as the root mean square of movement in 3d space.
There are many other variables listed that you might find useful, but for which we haven’t developed hard guidelines for. For instance, voxel SNR (timecourse SNR calculated for each individual voxel) is more dominated by scanner noise than slice SNR and might be useful. So might be max or mean relative motion, which is movement relative to the previous timepoint as opposed to the first timepoint.
The top left map is the voxel SNR map. This is formed by dividing the mean map (middle map) by the standard deviation (right map). Generally, it will take some practice in looking at these maps to tell what is ‘normal’, and this varies by coil. For the 12ch, you generally want the mean and SNR map to be uniform. For the 32ch, these are generally brighter at the edges of the brain.
The next row starts with the slope map. This calculates changes overtime, and therefore can be useful in highlighting movement. Areas of white or black have large movement.
Next (left graph) is the mean slice intensity over time, and you want these lines to be fairly smooth.
The right graph is the motion parameters from mcflirt in FSL. Here you also want relatively small changes. Make sure to check the axis scale closely to interpret the magnitude of the movements.
The last map is the mask, which you want to generally be uniform and covering the whole brain. If there are areas of drop out, this might suggest regions of low signal.
This tab shows a few more measurements than the main summary page, in addition to everything on the summary page.
In this tab, you can record overall ratings of the data, which can later be read by other users or can be used to filter ‘usable’ data in the database. There are several criteria by which data can be classified. These include motion, head coverage, signal homogeneity, ghosting, and RF noise/spiking. Motion can be assessed with the quantitative values that come out of the extended BOLD qc, while the rest need to be assessed via qualitative inspection of your data. Then, there is a final overall QC assessment rating that can be set.
This contains the scripts that were run in case you want to recreate something on your own.
This is where the files created are listed. You can click on any of these to download the files. Currently, these download with extra information on the beginning and end of the file name. We are working to correct this. The last two are the report files. The auto_report.txt file contains the statistics from the extended BOLD QC that can be used to qualify data. The slice report contains mostly the slice-based SNR measurements.
This might be necessary if you are scanning a non-typical protocol, such as on children, patient populations, or with the 32ch coil. The easiest way to create new standards is to look at the values across all of your subjects in an experiment. You can calculate the average value for the important parameters discussed above and then look for ‘outliers’. You can do this by recording key parameters manually, or by grabbing the summary report files via the command line (coming soon!), or by clicking on the report files from the ‘all files’ tab or on the left hand side of the man page. Currently, these download with extra information on the beginning and end of the file name. We are working to correct this. See below for how to interpret the files.
The auto QC report contains the parameters in the parameters box on the left, in very cryptic names.
qc_sSNR = slice SNR
qc_vSNR= voxel SNR
mot_rel_xyz_mean = Mean Rel Motion
mot_rel_xyz_max = Max Rel Motion
mot_abs_xyz_mean = Mean Abs Motion
mot_abs_xyz_max = Max Abs Motion
mot_rel_xyz_1mm = Movements > .1mm
mot_rel_xyz_5mm = Movements > .5mm
The slice report contains the slice SNR, slice mean, slice stdev, slice mean min and max across all timepoints, and the number of timepoints considered outliers. These are reported separately for each slice, and then averaged together in the end. This is where the slice SNR displayed in the parameters box comes from.
Click here for a video on how to use FSL to perform quality control. The datasets used are considered 'good' and are artifact free. Soon we will have additional videos demonstrating possible artifacts.
Click here for a manual created by Natasha Hansen et al. (email@example.com). This manual is a great starting place for learning about QC. It provides mild, moderate, and sever examples of all of the artifacts you want to look for in your data. Soon we will also have videos of these in the qualitative QC area.
MRI Qualitative Quality Control Manual, (2013). Natasha Hansen, Garth Coombs, Thilo Deckersbach, and Randy Buckner
Qualitative QC involves looking at your data. We recommend doing this via FSL (see video below on how to use). This does not mean you need to analyze your data in FSL, as the viewer will read any nifti files. After you finish the qualitative QC you can enter your findings into XNAT (CBSCentral) using the manual tab. For a better description of the various artifacts, see the QC manual.
- Load your structural scan. If you are using fsl, type:
fslview structural_name &
Scroll through all the slices in each view of the brain.
- Verify that that all of the brain is there (not clipped)
- Verify that if any part of the anatomy fell outside of the field of view and is wrapping around, that it doesn't overlap with the brain.
- Look for artifacts in the ‘ringing’, ‘striping’, and ‘blurring’ category. These artifacts can sometimes be easier to see if you use “Single View” (Tools > Single) to enlarge the view of the axial plane.
- Look for radio frequency (RF) noise, which resembles TV static over the scan image, and spiking, which looks like rigid stripes over the brain and/or background.
- Look for susceptibility artifacts (a black area of signal loss surrounded by bright and dark ripples - such as in the mouth when participants have a permanent retainer).
Turn on “Lightbox View” (Tools > Lightbox) and scroll down to slices that show the full walnut shape of the brain. Decrease the contrast by lowering the max brightness value while leaving the minimum brightness as the default value (0). Lower the max brightness to 1/3 the default value. Scroll through all the slices:
- Look for ghosting (a fainter displaced copy of the head, brain, or eyes).
- Pay attention to where the ghost image might overlap with the brain.
- Look for RF noise. With this contrast, a little RF noise (TV static) is normal, but you should check if there is more than usual. You should not see rigid stripes (spiking).
Reset the brightness values to default. Then increase the contrast by raising the minimum brightness value while leaving the maximum brightness value at the default. Raise the minimum brightness to 1/3, then 1/2, then 2/3 of the maximum, each time scrolling through all the slices in each view of the brain.
- Look for the homogeneity of the signal intensity. You will get to know what looks normal for your scanner and coil, but in general the outer rim of the brain is brighter than the interior. The signal homogeneity pattern should be symmetrical, particularly left to right. In general, it should also be symmetrical front to back, but if you use the 32 channel coil this will depend on how the subject’s head was padded.
Qualitative QC involves looking at your data. We recommend doing this via FSL (see video below on how to use). This does not mean you need to analyze your data in FSL, as the viewer will read any nifti files. After you finish the qualitative QC you can enter your findings into XNAT (CBSCentral) using the manual tab. For a better description of the artifacts, see the QC manual.
Load your functional data. You will have to perform QC for each BOLD run. If you are using fsl, type:
fslview functional_name &
Check for head coverage, make sure all of the brain target area (the part you care about for your study) is covered and not clipped by the edge of the field of view.
- Remember to scroll through all the slices in the coronal plane to check if the bottom tips of the temporal lobes are cut off.
Turn on movie mode. This will automatically cycle you through the timepoints (volumes), so you will be more likely to spot issues that are timepoint-specific as you scroll through the slices. Continue to scroll through the slices, making sure that movie mode plays through a full cycle of volumes and starts over at zero (0) several times.
- Look for motion slice artifact (sudden horizontal light and dark stripes through the brain that may only be there for particular timepoint(s).
- Look for RF noise (like TV static) and spiking (sudden extreme static and/or stripes through the brain). Mild-moderate degrees of RF noise can be more easily seen when you decrease the contrast (see step 7 below).
Turn on “Lightbox View” (Tools > Lightbox) and increase the view size to 200%.
- Scroll though all the slices in each view of the brain.
Decrease the contrast by lowering the maximum brightness value to 1/3 the default while leaving the minimum value at the default (0). Scroll through the slices in each plane:
- Check for ghosting (a fainter displaced copy of the brain). There will always be some amount of ghosting, but check how visible it is and is if it overlaps the original brain image.
- Look for RF noise and spiking again.
While leaving on lightbox and movie mode, reset the contrast to its default values. Then increase the contrast by raising the minimum brightness value while leaving the maximum at the default. Raise the minimum brightness to 1/3, then 1/2, then 2/3 of the maximum, each time scrolling through all the slices in each plane.
- Look for the homogeneity of the signal intensity. You will get to know what looks normal for your scanner and coil, but in general the outer rim of the grain is brighter than the interior. The signal homogeneity pattern should be symmetrical left to right. In general, it should also be symmetrical front to back, but if you use the 32 channel coil this will depend on how the subject’s head was padded.
We have a new piece of hardware thanks to the HCP project, called FIRMM, that will monitor movement in real-time. This is still a new product, so it can be a little finicky! The steps below must be followed in the EXACT order.
How to Use
- Get on the VPN so that you can access the NCF network. People have been using the iMAC in the control room for this, as it has an ethernet connection. Otherwise you can get some annoying lag with the motion monitoring.
- The Cisco Annyconnect is on there, but make sure to use your own Open Auth Token.
- Make sure the iMAC is connected via ethernet. The yellow cable should be plugged in, and you need to check that it is active. This computer is also used for eye tracking, so sometimes the ethernet is disabled.
- Open System preferences -> Internet, and make sure the location is set to 'Automatic'
- Register your subject
- Uncheck Load Program from Queue, if you forget, you need to delete everything from the left side after registration.
- Start FIRMM on the scanner.
- Use ctrl+escape to bring up the windows start menu.
- Right click on FIRMM_session_start, and select Run as administrator
- This will pop up a window, after it goes away, you can put things in the queue and should run your localizer and AA scout
- While this runs, login to the hardware from the iMAC.
- From a terminal window, and with the VPN running, open an ssh session.
- The password is written on a post-it on the computer.
- Once you are logged in, launch the program
- This will spit out some lines, and then eventually bring up the GUI, that looks a lot like a webpage.
- Start running your BOLD scan.
- The window on the far right that displays your slices should automatically show up. If it doesn't, you can open it by clicking on the Inline Display button on the far right of the window that shows your protocols. You can hover the mouse over the buttons if you are unsure of which one it is (second one from top).
- Once your DICOM's appear (usually takes a few seconds), you can press START on the FIRMM GUI.
- You only have to push this once. Additional BOLD runs will show up in the summary window, and the trace will restart automatically.
- When you are done, you want to close everything on the iMAC.
- Close the FIRMM GUI window (usually red x in top left corner), and then type exit
- Quit the VPN
Getting the data if you want it
The files are just a csv file consisting of 6 columns, one for each motion direction (3 translations, 3 rotations), from the motion tracking it does. You can sftp to the machine with the same address and login information. Username: firmmproc, Address:ncf-firmm1.rc.fas.harvard.edu. Then go to the directory: cd FIRMM/FIRMM_logs, and get your log file.
How to Use
You can see an example of the output in the image below. The blue line is a running trace of what the estimated frame wise displacement is. This is the relative motion from frame to frame, combined over the 3 translation and 3 rotation directions. There is a shading of green, yellow, and red, that corresponds to less than .2, .3, and .4mm respectively of total motion. These ranges are currently hard coded.
HCP uses some suggested metrics:
- Cancel the scan after the first 60sec if less than half of the initial frames have less than .4mm.
- You can tell if this criteria is met by looking in the summary area, this updates in real-time.
- Part of the thinking behind this is that if they moved that much in the beginning, they most likely they also moved during the reference scans used in the SMS protocols, which can hurt the general quality of your entire run. Depending on your experiment, this may not be possible.
- Give your participant feedback. When the run completes, you will get summary statistics in the bottom summary panel. If less than 75% of the frames have movement less than .3mm, this is considered a 'High Motion' run. Feel free to determine your own thresholds and terminology for your participants, just be consistent.
After every BOLD run you should scroll through your slices and look for movement, changes in ghosting, or changes in background luminance which could be spiking or RF interference. To do this, click on the Viewing tab on the far right side of the screen. In this window, you should load your BOLD images. Do this by bringing up the patient browser window by clicking on the image of the yellow folders in the lower right panel.
Navigate to your subject and then double click on the BOLD run you want to view, or drag it from the browser into the viewing window. You should see all the timepoints load.
Next, you want to increase the background contrast to make it easier to see the ghosting and background noise. Do this by holding the middle button on the mouse and moving the mouse down and to the left. You want them to look something like the image below.
Once you have the contrast turned up, you should be able to see some ghosting (red arrow), which is normal. You want to make sure this doesn't change much as you scroll through the images (by clicking on the up and down arrows on the right). In addition, you can look for changes in the background (the space outside of the brain). Sometimes you will see stripes appear. This is often a sign of RF interference or scanner spiking. You should also keep an eye on movement as you scroll through the images. See the image below for an example of what RF interference looks like. Please see here for what to do if something looks abnormal.
When you are done, you should close the loaded images. This way the next users won't see them, and it prevents the computer from becoming overloaded, which can happen if you load too many images at once. Do this by clicking on the close patient icon shown below.
There are several basic steps you can do to try and improve the quality of your data at the scanner.
- Movement: Provide feedback to your subject about their movement, if your paradigm permits. In general, it can be useful to provide gentle reminders that they need to stay still, or you can specifically tell them that you checked their movement and you noticed that they moved at a particular time (such as, the beginning, middle, or end of scan). This can allow the subject to realize what kind of movements aren't allowed. However, if you are studying emotions, or working with a group of participants that might not respond well to this feedback, then you may want to consider saying nothing or providing very general, generic feedback.
- Ghosting: There is generally not a great solution for this. However, if you see sever ghosting or a change in ghosting across the run, Ross (firstname.lastname@example.org) should be informed of the date, subject number, and run number where something was observed.
- RF/Spiking: One common source of RF interference is the button box. Make sure the subject is holding the box down by their side and not up in the scanner bore near their head, and make sure they are keeping it stationary. This should also be reported to Ross (email@example.com) with the date, subject number, and run number where observed.
It is important to make sure that every part of the brain you want to collect information about is inside the field of view (yellow box).
For the structural, this means making sure the top, bottom, and sides of the brain are in the yellow box, and that the nose isn't being cut off. Anything that falls just outside of the yellow FOV will 'wrap around' to the other side. For example, the noise would show up in the occipital lobe. If you use autoalign, then this will usually come up correctly for you, but it is always wise to look closely.
If you need to move the box, put your cursor over the center yellow circle, and your cursor will turn into the symbol shown below. To rotate the FOV, move the cursor to the edge of the box until you get the double arrows shown below. If you move the box in any other way, you will change the FOV. This is bad, as it can do things like change the voxel size.
For the functionals, setting the FOV (yellow box) is more complicated because it depends on the area of the brain you are interested in for your study. There is a trade off between voxel size, number of slices, and TR, so often you don't have as many slices as necessary to cover all of the brain you want. If you are lucky enough to have plenty of slices and use autoalign, then you might not have to do anything. Regardless, you should visually check that the yellow box covers all the 'important' parts of the brain. This can be done by loading the structural scan up and looking at your slices on top of it. See step 4 of conducting a typical scan for more details on this.
If your FOV is small and you are worried about getting everything, you might want to run a test scan, with just 2 or 3 TRs to see what your actual BOLD images look like. See here for how to change the number of timepoints.
If you do need to move the FOV, you will have to copy this new position to all other subsequent scans that should be using the same slice prescription (other BOLDS, or fieldmap). Please see here for how to move your slices after autoalign doesn't work.