mr

package
v0.0.0-...-57410b3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 14, 2024 License: MPL-2.0 Imports: 15 Imported by: 0

Documentation

Index

Constants

View Source
const (
	MapPhase      = "Map"
	ReducePhase   = "Reduce"
	FinishedPhase = "Finished"

	MaxRequestWaitingTime = 3 * time.Second
	MaxTaskWaitingTime    = 10 * time.Second

	TempFilePath = ""

	FilePath = ""
)

常量

View Source
const (
	OK       = "OK"
	ErrNoKey = "ErrNoKey"
)

Variables

This section is empty.

Functions

func MapTask

func MapTask(mapFunc func(string, string) []KeyValue, task Task, fileName string)

MapTask 执行 Map 任务返回中间文件位置

func ReduceTask

func ReduceTask(reduceFunc func(string, []string) string, task Task)

ReduceTask 执行 Reduce 任务

func Worker

func Worker(mapFunc func(string, string) []KeyValue, reduceFunc func(string, []string) string)

Worker main/mrworker.go calls this function. 创建一个 Worker 进程,等待指令 通过 RPC 请求任务;等待机制,让 Worker 不断询问还是由 Coordinator 通知 读文件并运行任务 - 得到的格式,任务类型和文件位置 使用encoding/json写读中间文件 完成任务通知 Coordinator,任务完成状态和位置

Types

type Args

type Args struct {
	Task Task
}

Args RPC 参数

type ByKey

type ByKey []KeyValue

func (ByKey) Len

func (a ByKey) Len() int

Len for sorting by key.

func (ByKey) Less

func (a ByKey) Less(i, j int) bool

func (ByKey) Swap

func (a ByKey) Swap(i, j int)

type Coordinator

type Coordinator struct {
	// contains filtered or unexported fields
}

Coordinator 管理和调度任务,包括划分任务,管理任务状态 同时 Coordinator 需要知道 Worker 的状态 队列机制使用 channel 实现 任务包括 2 阶段,Map 和 Reduce,先 Map 后 Reduce

func MakeCoordinator

func MakeCoordinator(files []string, nReduce int) *Coordinator

MakeCoordinator 输入文件数,为每一个文件 create a Coordinator. main/mrcoordinator.go calls this function. NReduce is the number of reduce tasks to use.

func (*Coordinator) Done

func (m *Coordinator) Done() bool

Done main/mrcoordinator.go calls Done() periodically to find out if the entire job has finished.

func (*Coordinator) FinishTaskHandler

func (m *Coordinator) FinishTaskHandler(args Args, reply *FinishTaskReply) error

func (*Coordinator) GetTaskHandler

func (m *Coordinator) GetTaskHandler(args Args, reply *GetTaskReply) error

GetTaskHandler RPC handlers for the worker to call.

type Err

type Err string

type FinishTaskReply

type FinishTaskReply struct {
	Reply
}

type GetArgs

type GetArgs struct {
	Key string
}

type GetReply

type GetReply struct {
	Err   Err
	Value string
}

type GetTaskReply

type GetTaskReply struct {
	Reply    Reply
	Task     Task
	FileName string
	NReduce  int
	MMap     int
}

type KV

type KV struct {
	// contains filtered or unexported fields
}

func (*KV) Get

func (kv *KV) Get(args *GetArgs, reply *GetReply) error

func (*KV) Put

func (kv *KV) Put(args *PutArgs, reply *PutReply) error

type KeyValue

type KeyValue struct {
	Key   string
	Value string
}

KeyValue Map functions return a slice of KeyValue.

type PutArgs

type PutArgs struct {
	Key   string
	Value string
}

type PutReply

type PutReply struct {
	Err Err
}

type Reply

type Reply struct {
	State int
}

Reply RPC 返回值

type Task

type Task struct {
	TaskType string
	// Id 用于对应文件名
	Id int
}

Task 任务信息

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL