doss · 2021年11月28日 0

Achieve Data storage service -1

Start by linking to a series of articles where we can view the catalog to explore distributed object storage. Let's go back to our theme 🙂

A data storage service is a service that stores object data, and it is a service that ultimately falls into the market. Our distributed object storage system is mainly exposed to HTTP services. The object data (pictures, video, text, etc.) are stored on local disks of local service nodes, although the disks of our service nodes can be NAS disks, etc., and can also be RAID1, RAID5, RAID10, etc. This is not the focus of this article, interested friends can search the Internet for relevant information to learn.

The system is mainly in the golang language, golang language has its powerful functions and minimal api, of course, other language developers can also implement their own according to this idea. All articles we will introduce some syntax and use of our golang code, so that you can easily get started with the golang language.

Let's explore how data services are implemented.

Reflecting before us is the main function, which is the entry function for all systems, and the startdatanode.go code is as follows:


The package main //main function must be under the main package, and each individually runable module needs to have a main package

import (//import dependency package, this is nothing to say.)
   "com.mgface.disobj/common"
   . "com.mgface.disobj/datanode/command"
   "github.com/urfave/cli"
)

Const usage' //define constant, note that this is an anti-single quote that can support multiple lines, java13 supports multi-line strings, using "" three double quotes.
 *
 The data service mainly provides storage object data
 *
`

func main() {
   The terminal command that is executed
   cmds := [] cli.Command{
      StartDNCommand,
   }
   common. RunFn(usage, cmds)
}

Here's the cli package, which is primarily a package that handles input parsing on the command line. Inside the main function is an array of Command below the cli package, which contains the StartDNCommand object. The type of the object is the Comend type.
Let's take a look at the true face of the ArtDNCommand object:)

var StartDNCommand = cli.Command{
   Name: "start",
   Aliases: [] string{"st"},
   Usage: 'Sample instruction:
      startdatanode start -na 127.0.0.1:5000 -mna 127.0.0.1:3001 -sdp C:\objects
      `,
   Flags: **startflag**,
   Action: func(ctx *cli.Context) error {
      var nodeaddr, mna, storedatapath, podnamespace string
      ------- parsing k8s env
      cfg := new(EnvConfig)
      env. IgnorePrefix()
      err := env. Fill(cfg)
      if err != nil {
         log. Info ("Not read to env environment...")
      } else {
         nodeaddr, mna, storedatapath, podnamespace = readEnv(cfg)
      }

If pns are empty, it is not a k8s environment
      if podnamespace == "" {
         nodeaddr, mna, storedatapath = readCli(ctx)
      }

Initialize the data
      api. Initval(storedatapath, nodeaddr)
      Start the service
      server. StartServer(nodeaddr, mna, podnamespace)
      return nil
   },
}
var **startflag** = [] cli.Flag{
   cli.StringFlag{
      Name: "na", //node address
      Usage: Node Address,
      Required: false,
   },
   cli.StringFlag{
      Name: "mna", //metanode address
      Usage: "Metadata service node address. Just configure a seed node that can connect properly."
      Required: false,
   },
   cli.StringFlag{
      Name: "sdp", //store data path
      Usage: "Data store path,"
      Required: false,
   },
   The todo profile is handled separately
   cli.StringFlag{
      Name: "file",
      Usage: Profile,
      Required: false,
   },
}

Here's the explanation of cli. The name of the parameter in Command:

The name of the argument Parameter name resolution
Name Directive name, we call it "start" here.
Aliases The alias of the instruction, usually shorthand, is called "st" here.
Usage How to use this directive, we use here "startdatanode start -na 127.0.0.1:5000 -mna 127.0.0.1:3001 -sdp C:objects"
Flags The set of parameters of the instruction, here we have these parameters na, nma, sdp, file
Action A callback action that is executed by an instruction, an action that we trigger after the execution of the instruction.

If you don't use cli. Flag, cli.Command these structures don't matter, see how the official website is introduced. Not familiar with how to use is not tight, because learning a new language is to follow the gourd painting, and then to study.

Let's focus on Action's code:

func(ctx *cli.Context) error {
var nodeaddr, mna, storedatapath, podnamespace string
------- parsing k8s env
cfg := new(EnvConfig)
env. IgnorePrefix()
err := env. Fill(cfg)
if err != nil {
log. Info ("Not read to env environment...")
} else {
nodeaddr, mna, storedatapath, podnamespace = readEnv(cfg)
}

If pns are empty, it is not a k8s environment
if podnamespace == "" {
nodeaddr, mna, storedatapath = readCli(ctx)
}

Initialize the data
api. Initval(storedatapath, nodeaddr)
Start the service
server. StartServer(nodeaddr, mna, podnamespace)
return nil
},

Let's first read the env variables of the system to see if they are set through the k8s pod, and if the podnamespace variable is empty, we don't have a k8s cloud platform. And then we're going to initialize the data,
Start the data storage service. Next, we'll dissect the source code to see how it works.

Source code address here https://gitee.com/wanyuxiang000/distributedObjStorage.git,welcome to pull down to read.