Team 3 :: Nick, Katy, Tarunya and Mike will be working on project #22: Creating a system to capture (and disseminate quickly/effectively) a larger set of assessment data points on student (and Teacher) performance.

    Link to NTLB 2010 Homepage

    Wiki Reflection Activity:

    Find a way to represent your team's design process and turn it in with your reflection sheet. If it makes sense, include process phases.

    Generate 3 or more anecdotes about your team's process that you would love to share will help teach your grandchildren design thinking when take a d.school classes. Example: In order to test how people act when they are anxious, we had people drink a 2 litters of water and wouldn't let them use the restroom until they answered 2 trivia questions (true student example about prototyping an experience with limited resources).

    Individually, please add 3 elements to each category of the I like, I wish, how to wiki page.

    Download file "Try2_Page_2_katy.jpg" (team agreement)

    Prototype Photos

    For April 27th class:
    The 2-3 Analogous Situations we explored and what we learned:
    1) Google Employee Performance Reviews
    • Transparency — Transparency in performance reviews and expectations varies a lot across business units. To address this, Jeanne’s group developed a “Skills matrix” that laid out the behaviors expected at each level for each performance level. The matrix was then used in reviews to explain performance ratings, and also as a guideline for developing an action plan for each employee, based on which areas in the matrix received lower ratings. One criticism is that the skills matrix was still fairly generic (e.g., consistent communication vs.
    • Static view and forward looking — Employees received bi-annually formal feedback from managers that rated their current performance along several dimensions. When they met with managers, they also developed an “Action Plan” for how to get to the next level.
    • Objective and subjective feedback combined — Managers rated employees on certain dimensions on a 1.0-5.0 scale. Note – they did not communicate the numbers in reviews, but instead had phrases that the #s translated too — e.g., 3.0 = “Meets expectations”. Also, verbal feedback on strengths and areas to work on was gathered from 3-5 people to get a 360-degree perspective. The employee proposed the people to write these reviews; the manager reviewed the list and changed it as they saw fit; some groups shared the direct quotes of feedback and in other areas managers consolidated and paraphrased the feedback, into one coherent review that anonymized the reviewers.
    • Exponentially tied to rewards — Performance reviews provide exponential upside. E.g., 3.0 means you get 100% of target bonus, 3.2 means 104%, 3.8 means 134%
    • Fit to a curve — Google aims to have an average review score of 3.4
    2) Zagat Restaurant Rating Process and Data Management
    Duel-pronged rating system:
    - Users generate "ratings" by giving restaurants a numerical score from 0-3 along different dimensions (food, service, etc.) - this score is converted into the ratings we see by an advanced computer algorithm
    - Zagat staff (including statisticians) keep records of past reviews and check factual accuracy of new and old reviews (i.e. they know the ins and outs of the data for their respective geographies)
    Potential translation for our project: Teachers as "users" who collect test/other current student data and schools as "Zagat staff" who maintain more factual/demographic/past performance data on students

    Data usage and management:
    - Data received through online customer surveys and subsequently compiled into the paper and online guides we associate with Zagat
    - Deeper use for data along demographic lines/trends is sold through B2B channel (e.g., First Republic Bank wants to know what restaurants resonate with single women in mid-30s with income of $60K or higher to use to target clients)
    Potential translation for our project: Need to understand not just higher level trends but individual student trends to target areas of strength and relative weakness

    Process issues:
    - Surveys are time consuming and after 8 screens of questions, 70% of people abandon the survey
    - Subscribers to Zagat are representative of who Zagat used to target but not who it wants to target (i.e. they need to broaden base of raters to represent new clientele)
    Potential translation for our project: 1) Time is a huge issue for teachers 2) Our product can't just self-select teachers who are comfortable with technology but must have mass appeal in order to be effective

    3) Chevron Oil Refinery Assessment Systems

    A) Assessment Overview:
    Chevron uses a very complex, quantitative system to measure physical and chemical elements within their refineries. Tens of thousands of electronic and mechanical sensors are constantly reporting information of the on-goings within the refinery. The engineers have developed complex computer algorithms to analyze all of the inputs. Assessment is essential for ensuring that the plant runs efficiently, profitably and safely. There are too many sensors reporting data for humans alone to analyze the constant stream of information. The engineers must rely on computers to process the data and report relevant findings to the engineers so that they can manage the refinery appropriately. Beyond the traditional monitoring of the current conditions and operations of the plant, the computers use complex algorithms to analyze the historical trends to both highlight red flags and required interventions as well as predictive

    B) Historical Analysis - Red Flags & Interventions:
    From an historical standpoint, Chevron is able to tell which components or even systems within the refinery have been causing multiple problems and may be in need of an intervention or overhaul. From a classroom perspective, a system of like this may be able to warn teachers that a student has been delinquent in multiple classes so that they can take steps to intervene in the problem, rather than waiting for one teacher to observe multiple instances of delinquency before intervening.

    C) Predictive Assessments:

    Understanding how components and stems function in relation to one another helps engineers predict where issues may emerge so they can take preventative measures. From a classroom perspective, teachers could be warned that because kids struggled with concepts X and Y, it is likely that they will struggle with concept Z (but not necessarily concept A), so the teacher should spend more time reviewing concept Z.

    Feedback on our rough prototype:

    What we learned about the user from our rough prototype:

    The user wants feedback and transparency. E.g., Assessment tool should help teacher understand where students are AND it could be used to provide feedback and transparency to students for where they are.

    Our most current POV statement (descrip of user) needs ______ (think verb) b/c ______ (insight). :

    [INSERT TEACHER NAME HERE] is a data-starved, moderately tech-savvy, enthusiastic teacher who wants an easy and engaging way to increase the breadth and access of real-time and cumulative assessment data for his/her students so that he/she can better tailor curriculum to match student needs, identify struggling students and move through material at the appropriate pace.

    Key Insights

    1) Technology and Time hurdles prevent teachers from using this concept

    2) Need to use technology to quickly distill key insights for teachers

    3) Need to facilitate the collection of data to reduce burden on Teachers

    4) Coordinating students to input assessment data is a complex process

    5) Student assessment data can be uses to track teacher effectiveness

    Areas for Help:

    1) How would you use this in your classroom? (Pre-tests? Interim Assessments? Private vs. Public results? Bank of Samples vs. Customizable?) What are the limitations for you using this more?

    2) What level of time/technology hurdle is acceptable?

    3) What are you biggest frustrations or roadblocks in doing more assessments?

    ideas/components/aspects we are going to prototype on Thursday, April 29th:

    - Rubric/Skills matrix to structure multiple aspects of assessment -- And how does assessment tie to curriculum?
    - Using assessment to group students -- in common ability level? matching up students that have mastered material with students that need help?

    Teacher Quotes

    “We take a couple big assessments per year, but we don’t get the results until the year is over. The information is too old to really be useful and it comes in an illegible format – We just leave the CDs on a shelf to collect dust.”

    - Green Dot Administrator

    “I used to spend my lunch break grading quizzes from the morning so that I could group students more effectively – Most teachers would never think of making the time for more immediate assessments.”

    - Teach for AmericaTeacher

    Map of insights for teacher tracking device

    Insight map

    Business Plan class exercise

    Customers: *Teachers (end users)*, Schools (principals, CTOs, etc.), Districts, Foundations, Parents (PTA), DOE

    Key risks are ...

    Desirability - Teacher pain point might not be big enough;

    Feasibility - Creating a tool that meets 'holistic' child assessment approach

    Viability -Will data prove useful? Is real-time input practical?

    To be successful we must ...

    D - Convince customers that data on 'holistic' child is critical

    F - Hire talented programmers; Build connections with foundations and districts

    V - Application must be super intuitive, user-friendly; data analysis should be automated and useful

    Next steps are ...

    D -Determine size and scope of 'pain points'

    F -Determine market for programmers; Approach foundations

    V -Build prototype and test

    $500K over 6 months

    HMW enable teachers to easily gather more holistic and real time data on student behavior and engagement with minimal burden

    Mr/Mrs Teacher is a data-starved, time-constrained, enthusiastic teacher who needs an easy, real-time way to capture a more holistic view of student performance (engagement and behavior).


    Assessment empathy video
    Download file "100511-NTLB_Final 


    Nick Porfilio
    Apr 26, 2010


    Executive Searches

    Information Gathering: Depends on the situation but the majority of times the search firm already has information regarding the candidate in its own database that it has gathered from public sources and other less admirable ways. The candidate, when s/he becomes engaged with the firm, also provides information about his/her qualifications, past work experiences, etc. Some of this information is independently verified but most is not because of the time and effort it takes to do so but also because some information, especially from former employers, is not verifiable due to legal/privacy issues (non-verification has presented issues in the past).

    Limited Use: The information on the candidate is very important which is surprising it’s not really verified. However, while information provided on paper is useful, most offers are made based on face-to-face interactions with the potential employer.

    Subjectivity: There is no objective bar for candidates although some may believe other. The goal of any search firm is to match a candidate with an organizations search criteria. Sometimes the candidates the search firm feels are solid candidates do not get the jobs because s/he is just not the right fit for the organization.

    Conclusion: Information is important but it’s only the means not the end.

    Jennifer Carolan
    May 6, 2010

    Nick, Katy, Tarunya and Mike, Im writing with your advisor/mentor information. Your advisor is Michael Berman, founder of east Palo alto phoenix academy. Please go ahead and contact him to schedule your first meeting. His email is michael.berman@epapa.org. He has some time next week. He will be a fantastic advisor. Email me with any questions. Have a nice weekend. Jennifer

    /groups/k12/search/index.rss?tag=hotlist/groups/k12/search/?tag=hotWhat’s HotHotListHot!?tag=hot6/groups/k12/sidebar/HotListAdded tag - hotdyoungDevon Young2016-06-09 18:05:15+00:002016-06-09 18:05:15addTag14Added tag - activitydyoungDevon Young2016-06-09 08:51:57+00:002016-06-09 08:51:57addTag13Added tag - testdyoungDevon Young2016-06-09 08:51:48+00:002016-06-09 08:51:48addTag12Added tag - prototypedyoungDevon Young2016-06-09 08:51:46+00:002016-06-09 08:51:46addTag11Added tag - ideatedyoungDevon Young2016-06-09 08:51:45+00:002016-06-09 08:51:45addTag10Added tag - definedyoungDevon Young2016-06-09 08:51:43+00:002016-06-09 08:51:43addTag9Added tag - empathizedyoungDevon Young2016-06-09 08:51:42+00:002016-06-09 08:51:42addTag8Added tag - stokedyoungDevon Young2016-06-09 08:51:40+00:002016-06-09 08:51:40addTag7Added tag - design thinking processdyoungDevon Young2016-06-09 08:51:38+00:002016-06-09 08:51:38addTag6Added tag - resourcesdyoungDevon Young2016-06-09 08:51:35+00:002016-06-09 08:51:35addTag5dyoungDevon Young2016-06-06 23:47:53+00:002016-06-06 23:47:53updated4dyoungDevon Young2016-06-06 21:53:23+00:002016-06-06 21:53:23updated3dyoungDevon Young2016-06-06 20:58:40+00:002016-06-06 20:58:40updated2First createddyoungDevon Young2016-06-06 20:58:21+00:002016-06-06 20:58:21created1wiki2016-06-06T23:47:53+00:00groups/k12/wiki/bb07fFalseResources and Tools by Stage/groups/k12/wiki/bb07f/Resources_and_Tools_by_Stage.htmlDevon Young14 updatesResources and Tools by Stage Key tools and worksheets in each step Warm-up [Download file "Stoke Deck.pdf"] Improv Toolkit for Educators, Improv Activiti...Falsedyoung2016-06-06T23:47:53+00:00Added tag - hotdyoungDevon Young2016-06-09 18:04:48+00:002016-06-09 18:04:48addTag18Added tag - full-cycledyoungDevon Young2016-06-09 06:21:38+00:002016-06-09 06:21:38addTag17Added tag - designed by k12 labdyoungDevon Young2016-06-09 06:21:34+00:002016-06-09 06:21:34addTag16Added tag - educator workshopdyoungDevon Young2016-06-09 06:21:23+00:002016-06-09 06:21:23addTag15Added tag - 2-hourdyoungDevon Young2016-06-09 06:21:18+00:002016-06-09 06:21:18addTag14Added tag - 2016dyoungDevon Young2016-06-09 06:21:08+00:002016-06-09 06:21:08addTag13Removed tag - workshopdyoungDevon Young2016-06-09 06:21:02+00:002016-06-09 06:21:02removeTag12Removed tag - morning routinedyoungDevon Young2016-06-09 06:21:00+00:002016-06-09 06:21:00removeTag11Removed tag - dp0dyoungDevon Young2016-06-09 06:20:59+00:002016-06-09 06:20:59removeTag10Removed tag - design thinking burstdyoungDevon Young2016-06-09 06:20:56+00:002016-06-09 06:20:56removeTag9Added tag - workshopdyoungDevon Young2016-06-06 15:58:59+00:002016-06-06 15:58:59addTag8Added tag - design thinking burstdyoungDevon Young2016-05-23 20:34:51+00:002016-05-23 20:34:51addTag7Removed tag - burstdyoungDevon Young2016-05-23 20:34:43+00:002016-05-23 20:34:43removeTag6Added tag - morning routinedyoungDevon Young2016-05-23 20:34:05+00:002016-05-23 20:34:05addTag5Added tag - dp0dyoungDevon Young2016-05-23 20:33:54+00:002016-05-23 20:33:54addTag4Added tag - burstdyoungDevon Young2016-05-23 20:33:47+00:002016-05-23 20:33:47addTag3dyoungDevon Young2016-05-23 16:46:27+00:002016-05-23 16:46:27updated2First createddyoungDevon Young2016-05-23 16:08:56+00:002016-05-23 16:08:56created1wiki2016-05-23T16:46:27+00:00groups/k12/wiki/213e5FalseDesign Thinking Burst: May 18, 2016 /groups/k12/wiki/213e5/Design_Thinking_Burst_May_18_2016_.htmlDevon Young18 updatesDesign Thinking Burst: May 18, 2016 What: An introductory experience for educators who are interested in learning about the design thinking process. When: April 28, 2016 at the d...Falsedyoung2016-05-23T16:46:27+00:00Added tag - hotdyoungDevon Young2016-06-09 18:05:46+00:002016-06-09 18:05:46addTag16Added tag - full-cycledyoungDevon Young2016-06-09 06:22:15+00:002016-06-09 06:22:15addTag15Added tag - designed by k12 labdyoungDevon Young2016-06-09 06:22:11+00:002016-06-09 06:22:11addTag14Added tag - educator workshopdyoungDevon Young2016-06-09 06:22:06+00:002016-06-09 06:22:06addTag13Added tag - 2-daydyoungDevon Young2016-06-09 06:22:01+00:002016-06-09 06:22:01addTag12Added tag - 2016dyoungDevon Young2016-06-09 06:21:58+00:002016-06-09 06:21:58addTag11Removed tag - workshopdyoungDevon Young2016-06-09 06:21:52+00:002016-06-09 06:21:52removeTag10Removed tag - discover design thinkingdyoungDevon Young2016-06-09 06:21:50+00:002016-06-09 06:21:50removeTag9Added tag - workshopdyoungDevon Young2016-06-06 15:59:14+00:002016-06-06 15:59:14addTag8Removed tag - savedyoungDevon Young2016-06-06 15:59:10+00:002016-06-06 15:59:10removeTag7Added tag - discover design thinkingdyoungDevon Young2016-05-23 21:41:41+00:002016-05-23 21:41:41addTag6Added tag - savedyoungDevon Young2016-05-16 20:35:23+00:002016-05-16 20:35:23addTag5dyoungDevon Young2016-05-13 21:14:20+00:002016-05-13 21:14:20updated4dyoungDevon Young2016-05-11 15:57:56+00:002016-05-11 15:57:56updated3dyoungDevon Young2016-05-11 03:15:39+00:002016-05-11 03:15:39updated2First createddyoungDevon Young2016-05-11 03:01:09+00:002016-05-11 03:01:09created1wiki2016-05-13T21:14:20+00:00groups/k12/wiki/7c78cFalseDiscover Design Thinking workshop: May 5-6 2016/groups/k12/wiki/7c78c/Discover_Design_Thinking_workshop_May_56_2016.htmlDevon Young16 updatesDiscover Design Thinking workshop: May 5-6 2016 Edit this page Create a new page Delete this ...Falsedyoung2016-05-13T21:14:20+00:00Added tag - hotarazAriel Raz2016-06-09 18:05:38+00:002016-06-09 18:05:38addTag13Added tag - prototypearazAriel Raz2016-06-09 06:30:10+00:002016-06-09 06:30:10addTag12Added tag - full-cyclearazAriel Raz2016-06-09 06:30:00+00:002016-06-09 06:30:00addTag11Added tag - grades 9-12arazAriel Raz2016-06-09 06:29:55+00:002016-06-09 06:29:55addTag10Added tag - student curriculumarazAriel Raz2016-06-09 06:29:51+00:002016-06-09 06:29:51addTag9Added tag - 1-weekarazAriel Raz2016-06-09 06:28:21+00:002016-06-09 06:28:21addTag8Added tag - 2016arazAriel Raz2016-06-09 06:28:18+00:002016-06-09 06:28:18addTag7Removed tag - student workshoparazAriel Raz2016-06-09 06:28:07+00:002016-06-09 06:28:07removeTag6Removed tag - fail forwardarazAriel Raz2016-06-09 06:28:03+00:002016-06-09 06:28:03removeTag5Added tag - student workshoparazAriel Raz2016-06-06 16:37:11+00:002016-06-06 16:37:11addTag4Added tag - fail forwardarazAriel Raz2016-06-06 16:36:53+00:002016-06-06 16:36:53addTag3arazAriel Raz2016-03-14 03:01:39+00:002016-03-14 03:01:39updated2First createdarazAriel Raz2016-03-14 02:48:24+00:002016-03-14 02:48:24created1wiki2016-03-14T03:01:39+00:00groups/k12/wiki/0502dFalseMt. Vernon Presbyterian School - Fail Forward/groups/k12/wiki/0502d/Mt_Vernon_Presbyterian_School__Fail_Forward.htmlAriel Raz13 updatesMt. Vernon Presbyterian School - Fail Forward On this page you'll find resources for the one week workshop we held with Mt. Vernon Presbyterian School. The frame of the weeklong experience was...Falsearaz2016-03-14T03:01:39+00:00weblog2009-11-19T16:29:14+00:00groups/k12/weblog/56457FalseNew Riverside-Nueva partnership/groups/k12/weblog/56457/New_RiversideNueva_partnership.htmlLeticia Britos0 updatesNew Riverside-Nueva partnership The Nueva School has some exciting news to report. After the six of us (Rebecca, Megan, Keely, Kim, Anna, and I) from Nueva met Deepa and Veena fro...Falselbritos2009-11-19T16:29:14+00:00hot/groups/k12/search/index.rss?sort=modifiedDate&kind=all&sortDirection=reverse&excludePages=wiki/welcomelist/groups/k12/search/?sort=modifiedDate&kind=all&sortDirection=reverse&excludePages=wiki/welcomeRecent ChangesRecentChangesListUpdates?sort=modifiedDate&kind=all&sortDirection=reverse&excludePages=wiki/welcome0/groups/k12/sidebar/RecentChangesListmodifiedDateallRecent ChangesRecentChangesListUpdateswiki/welcomeNo recent changes.reverse5search