Project

Profile

Help

Task #7487

Fold keys - replace hierarchy

Added by Ilfat Kinyaev over 4 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
High
Category:
performance
Sprint/Milestone:
Start date:
07.09.2016
Due date:
% Done:

0%

Estimated time:
1:00 h
Target branch:
Test scenario:

Run
cd scripts/scripts/benchmark_parser/
ruby -r './parse_log.rb' -r 'json' -e "hash = parse_sysbench_results_raw(File.read('./test_data/7487_flatten_sysbench_json')); puts JSON.pretty_generate flatten_keys(hash)"

Expected output:
{
"OLTP_test_statistics.queries_performed.read": 390600,
"OLTP_test_statistics.queries_performed.write": 111569,
"OLTP_test_statistics.queries_performed.other": 55787,
"OLTP_test_statistics.queries_performed.total": 557956,
"OLTP_test_statistics.transactions": "27887 (92.83 per sec.)",
"OLTP_test_statistics.read/write_requests": "502169 (1671.64 per sec.)",
"OLTP_test_statistics.other_operations": "55787 (185.71 per sec.)",
"OLTP_test_statistics.ignored_errors": "13 (0.04 per sec.)",
"OLTP_test_statistics.reconnects": "0 (0.00 per sec.)",
"General_statistics.total_time": "300.4046s",
"General_statistics.total_number_of_events": 27887,
"General_statistics.total_time_taken_by_event_execution": "9605.3034s",
"General_statistics.response_time.min": "306.27ms",
"General_statistics.response_time.avg": "344.44ms",
"General_statistics.response_time.max": "2474.96ms",
"General_statistics.response_time.approx._95_percentile": "397.97ms",
"Threads_fairness.events_(avg/stddev)": "871.4688/6.81",
"Threads_fairness.execution_time_(avg/stddev)": "300.1657/0.12"
}


Description

In ./scripts/benchmark_parser/parse_log.rb realize flatten_keys(hash):
Input: hash with split hierarchy.
Output: hash with flatten hierarchy.
Need "fold" keys - replace hierarchy (like http://www.convertcsv.com/json-to-csv.htm):

{
    "OLTP test statistics": {
        "queries performed": {
            "read": 389214,

to

{ 
"OLTP_test_statistics.queries_performed.read": 389214,

Given hash (slash at end shows end of one hash object):

SYSBENCH_RESULTS_RAW \
OLTP test statistics: \
    queries performed: \
        read:                            390600 \
        write:                           111569 \
        other:                           55787 \
        total:                           557956 \
    transactions:                        27887  (92.83 per sec.) \
    read/write requests:                 502169 (1671.64 per sec.) \
    other operations:                    55787  (185.71 per sec.) \
    ignored errors:                      13     (0.04 per sec.) \
    reconnects:                          0      (0.00 per sec.) \
 \
General statistics: \
    total time:                          300.4046s \
    total number of events:              27887 \
    total time taken by event execution: 9605.3034s \
    response time: \
         min:                                306.27ms \
         avg:                                344.44ms \
         max:                               2474.96ms \
         approx.  95 percentile:             397.97ms \
 \
Threads fairness: \
    events (avg/stddev):           871.4688/6.81 \
    execution time (avg/stddev):   300.1657/0.12 \


Related issues

Related to [mdbci] Maria DB Continuous integration tool - Task #7425: implement script for parsing sysbench log and collecting resultsClosed18.08.2016

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

History

#1 Updated by Ilfat Kinyaev over 4 years ago

  • Related to Task #7425: implement script for parsing sysbench log and collecting results added

#2 Updated by Mark Zaslavskiy over 4 years ago

  • Assignee set to Ilfat Kinyaev

#3 Updated by Mark Zaslavskiy over 4 years ago

  • Assignee changed from Ilfat Kinyaev to Alexander Kaluzhny

#4 Updated by Alexander Kaluzhny over 4 years ago

  • Status changed from New to Active / In progress

#5 Updated by Mark Zaslavskiy over 4 years ago

  • Priority changed from Normal to High

#6 Updated by Ilfat Kinyaev over 4 years ago

  • Description updated (diff)

#7 Updated by Alexander Kaluzhny over 4 years ago

  • Status changed from Active / In progress to Review
  • Assignee changed from Alexander Kaluzhny to Mark Zaslavskiy
  • Test scenario updated (diff)

#8 Updated by Alexander Kaluzhny over 4 years ago

  • Test scenario updated (diff)

#9 Updated by Alexander Kaluzhny over 4 years ago

#10 Updated by Mark Zaslavskiy over 4 years ago

  • Status changed from Review to Closed

Also available in: Atom PDF